Skip to content

Add snapshot selection to manifest inspection - #3913

Closed
emecii wants to merge 1 commit into
apache:mainfrom
emecii:feat/inspect-manifests-snapshot
Closed

emecii wants to merge 1 commit into
apache:mainfrom
emecii:feat/inspect-manifests-snapshot

Conversation

@emecii

@emecii emecii commented Sep 6, 2026

Copy link
Copy Markdown

Rationale for this change

Implements the manifest-inspection snapshot-selection part of #1053; the umbrella issue remains open.

table.inspect.manifests(snapshot_id=...) selects a retained snapshot using the existing lookup helper. Calls without an ID retain current-snapshot and typed empty-table behavior; unknown or expired IDs raise the existing lookup error.

Partition summaries use the selected snapshot's schema and each manifest's partition spec. When a retained manifest references a dropped source, resolve its field ID from retained schemas. This keeps old manifests readable after schema and partition evolution, including a reused column name with a different field ID/type. Snapshots without a schema ID follow the existing current-schema fallback convention; unavailable recorded schemas warn before falling back.

all_manifests() retains its signature and all-snapshot reference rows. It receives the historical summary correction through the shared generator. No general readable-metrics or bound-conversion changes.

Are these changes tested?

Real local catalog tests cover old/current selection, exact manifest paths, empty tables, invalid and expired IDs, renamed/dropped partition sources, mixed partition specs, reused names, null/empty bounds, type promotion, schema fallback, and all-manifest references.

  • Inspection suite: 21 passed.
  • Full local unit target: 4,028 passed, 3 skipped, 1,571 deselected.
  • make lint: passed.
  • Apache RAT: passed on a clean export of the commit.
  • Restoring current-schema-only decoding makes two historical-summary cases fail.

Spark/Docker parity, cloud suites, other Python/platform combinations and documentation build were not run. Historical fallback relies on retained schema metadata; it does not recover types from removed metadata files.

Are there any user-facing changes?

Yes: optional snapshot_id on InspectTable.manifests, documented in the API guide. Existing calls remain valid; historical partition summaries use retained schemas/specs.

AI assistance

OpenAI Codex generated the implementation, tests, documentation and PR draft, ran the reported checks, and reviewed the diff. The commit records Generated-by: OpenAI Codex. These statements describe agent work and do not assert personal human review.

Decode partition summaries with snapshot schemas and retained source fields so historical manifests remain readable after evolution.

Generated-by: OpenAI Codex
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that's incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

@github-actions github-actions Bot added the stale label Oct 7, 2026
@github-actions

Copy link
Copy Markdown

This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.

@github-actions github-actions Bot closed this Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant