[lucasfang]

Purpose

Linked issue: close #526

The current Table::NewScan/NewRead/NewWrite/NewCommit methods route back through context-based factories, so calling an operator on a loaded table does not bind it to that table's schema and storage identity. Table-kind dispatch is also concentrated in table.cpp, with format tables exposed through a separate handle.

Introduce a common table hierarchy so Catalog::GetTable() returns the appropriate managed, format or system table, and operator creation dispatches through that implementation.

• Add FileStoreTable with append-only and primary-key implementations, reusing the FileStore factories from refactor(table): extract file store factories #516 as merged into main. FileStore continues to derive its branch from options.
• Make FormatTable and SystemTable implement the common Table interface. System tables expose their own schemas and reject writes and commits.
• Keep loaded-table operators bound to their schema, path and filesystem while applying operation options and validating conflicting managed-table types, branches and write/commit identities.
• Route the existing static operator factories through TableFactory and the virtual operator methods. Preserve main's precise file-index evaluation behavior through both entrypoints.
• Retain catalog ownership for deferred global-system-table reads, and preserve URI routing, fallback-read options, system-table snapshot selection and mandatory changelog settings.

This is the hierarchy step of #459, following #516. Execution-context separation and general Table Copy/time-travel APIs remain separate follow-ups.

Tests

Full build and all 36 CTest targets passed on commit 0d2ddf52:

cmake --build build -j 40
ctest --test-dir build --output-on-failure -j 4

• Core: 2,682 passed; REST: 190 passed; read integration: 304 passed. Format-specific conditional skips remain in the suite.
• Focused hierarchy/read/index tests: 223 passed; system-table read integration tests: 32 passed.
• All applicable pre-commit checks passed for the 56 changed files; git diff apache/main HEAD --check passed.

Regression coverage includes catalog type dispatch, loaded schema/path/filesystem binding, explicit branch conflicts and aliases, builder retries after failed validation, deferred catalog lifetime, URI-map merging for main/fallback reads, changelog sequence numbers, system-table snapshot selection, and precise-index entrypoint parity.

API and Format

• Table becomes abstract, with a virtual destructor and virtual operator creation methods. Callers that directly constructed Table should load it through the catalog.
• Rename Table::LatestSchema() and FormatTable::LatestSchema() to GetSchema(), without compatibility aliases. The accessor returns the bound schema without metadata I/O; reload through the catalog to refresh it.
• Remove CurrentBranch() from the common Table interface; managed-table branch handling belongs to FileStoreTable.
• FormatTable derives from Table. Catalog::GetFormatTable() remains available for typed callers.
• The existing ContextBuilder and static operator Create() signatures remain unchanged. Catalogs must be owned by std::shared_ptr when loading global system tables, which retain the catalog for deferred reads.
• No storage-format or protocol changes.

Documentation

Update the catalog and format-table guides to describe common catalog dispatch, bound-table operator semantics, schema access, branch validation and global-system-table ownership.

Generative AI tooling

Generated-by: OpenAI Codex (GPT-6)

[SteNicholas]

@lucasfang, I left two comments for this introduction. PTAL.

[SteNicholas]

Bind format tables to the table-specific filesystem before returning them. When DATA_TOKEN_ENABLED is true and no filesystem was explicitly supplied, fs_ is the catalog-wide filesystem; only GetTableFileSystem(identifier) creates the RestTokenFileSystem that fetches and refreshes this table's credentials. This new format-table branch returns before that call. FormatTable::ForContext() preserves the bound filesystem, and its scan/read/write implementations access storage through it, so even supplying WithCatalog() on an operation context cannot recover the table token. On deployments where storage access requires the table-scoped credentials, GetTable() succeeds but subsequent file operations fail. Resolve the filesystem as the managed and system-table branches do, and add a format-table regression with data tokens enabled that verifies the token endpoint is used on storage access.

[SteNicholas]

Canonicalize identifier aliases of the main branch before adding the branch option. For Identifier("db", "tbl$branch_MAIN"), the catalog and Identifier::GetBranchNameOrDefault() resolve the table to main, but ResolveBranch() reads GetBranchName() and only normalizes blank strings, so this code inserts branch=MAIN. A context built with ScanContextBuilder(path).WithCatalog(catalog, identifier) then makes TableScan::Create() look for a schema under branch/branch-MAIN instead of the existing main schema; passing it to the catalog-loaded table's NewScan() instead fails with a branch contradiction. Before this change the scan builder did not inject that option and the same main-table scan worked. Canonicalize the identifier's main alias while preserving the case-sensitive semantics of explicitly named branch options, and cover both static and loaded-table scan entrypoints with $branch_MAIN.