Skip to content

Allow disabling remote signing in REST Catalog #3888

Open

Description

@BenCradock

Feature Request / Improvement

When using a REST Catalog (e.g. Lakekeeper), pyiceberg gets remote signing details from the catalog.

However, sometimes it would be useful to not use remote signing, in the case where the process performing the operations has access to the S3 endpoint through secret access keys. This is because remote signing requires a network round trip (plus any authorisation on server side).

It would be great if we could disable remote signing by respecting the s3.signer catalog property if it is empty.

The FileIO object gets created in pyiceberg/catalog/rest/__init__.py:

return Table(
    identifier=identifier_tuple,
    metadata_location=table_response.metadata_location,  # type: ignore
    metadata=table_response.metadata,
    io=self._load_file_io(
        {**table_response.metadata.properties, **table_response.config}, table_response.metadata_location
    ),
    catalog=self,
    config=table_response.config,
)

This doesn't use the catalog properties that have been passed to the REST catalog init as far as I can tell.

I might be wrong, but I don't think Support storage-credentials in REST catalog LoadTableResult- #3042
fixes this, because it still uses the response from the catalog.

Thanks!

Activity

  1. emecii commented on Sep 7, 2026

    @emecii

    I traced the current flow: RestCatalog._load_file_io() already merges self.properties, but the per-table response config is merged later, so a client-provided s3.signer="" cannot currently override a server S3V4RestSigner value.\n\nWould a narrow rule be acceptable: when the client explicitly supplies an empty s3.signer, apply that value after the table response solely as an opt-out; otherwise preserve today’s response-wins precedence? I would cover both Table and StagedTable construction paths and leave all other properties unchanged.

  2. BenCradock commented on Sep 8, 2026

    @BenCradock
    Author

    Yes that makes sense to me - happy for it to be only on an explicit empty s3.signer. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions