Repository navigation
Conversation
76e12b2 to
a300841
Compare
Surface the new `sseCustomerKey` field on `s3Credentials` through the ObjectStore CRD so users can enable Server-Side Encryption with Customer-provided keys (SSE-C). This is required by S3-compatible providers that only support SSE-C for encryption at rest, such as Hetzner Object Storage. The field flows through the embedded BarmanObjectStoreConfiguration from the barman-cloud library, so this change is limited to bumping the dependency, regenerating the CRD and the consolidated manifest, and documenting usage in the object stores guide. Depends on cloudnative-pg/barman-cloud#284 (temporarily pinned via a replace directive until that change is released). Closes cloudnative-pg#646 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Florian Schauer <florian@schauer.to>
|
FYI: barman release is out: https://github.com/EnterpriseDB/barman/releases/tag/release%2F3.20.0 |
|
Tested at
|
d097344 to
14f89b6
Compare
Surface the new `sseCustomerKey` field on `s3Credentials` through the ObjectStore CRD so users can enable Server-Side Encryption with Customer-provided keys (SSE-C). This is required by S3-compatible providers that only support SSE-C for encryption at rest, such as Hetzner Object Storage. The field flows through the embedded BarmanObjectStoreConfiguration from the barman-cloud library, so this change is limited to bumping the dependency, regenerating the CRD and the consolidated manifest, and documenting usage in the object stores guide. Depends on cloudnative-pg/barman-cloud#284 (temporarily pinned via a replace directive until that change is released). Closes cloudnative-pg#646 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Florian Schauer <florian@schauer.to>
|
Thanks for the test run and the validation change, @andrein — merged schaurian#1 into this branch. Updated the branch (
Full For the record, Barman 3.20.0 with |
Point the temporary barman-cloud replace at the head of cloudnative-pg/barman-cloud#284 that keeps each SSE-C key in its own file. A replica cluster archives WAL to its own object store while restoring it from the source one in the same sidecar, and with a single shared file each operation could run with the key of the other object store. Signed-off-by: Armando Ruocco <armando.ruocco@enterprisedb.com>
Also points the temporary barman-cloud replace at the head of cloudnative-pg/barman-cloud#284 that serializes the writes of the SSE-C key files. Signed-off-by: Armando Ruocco <armando.ruocco@enterprisedb.com>
Point the temporary barman-cloud replace at the head of cloudnative-pg/barman-cloud#284 that stops passing --sse-customer-key to barman-cloud-check-wal-archive. That command rejects the option, and it runs before the first WAL file is archived, so an object store with an SSE-C key could never start archiving. The SSE-C e2e test caught it. Signed-off-by: Armando Ruocco <armando.ruocco@enterprisedb.com>
|
I pushed a few commits on top.
The same shared-file problem exists for the Google credentials, fixed separately in #309. |
ac9c5a9 to
c6e4b16
Compare
7b0cad0 to
9ee8eac
Compare
Add an `sseCustomerKey` field to the S3 credentials that references a secret holding a base64-encoded 256-bit AES key. When set, the key is materialized to a file and passed to every barman-cloud command through the `--sse-customer-key file://` option, enabling Server-Side Encryption with Customer-provided keys (SSE-C). This is required by S3-compatible providers that only support SSE-C for encryption at rest (e.g. Hetzner Object Storage). The option is injected in the shared `appendCloudProviderOptions` chokepoint, so it applies to all barman-cloud-* commands (backup, wal-archive, wal-restore, restore, backup-list, backup-delete, check-wal-archive), and is orthogonal to the existing bucket-managed `encryption` (SSE-S3/SSE-KMS) option. The key is materialized before the auth-method branching so it works with every authentication method, including inheritFromIAMRole. Requires a barman release that ships the `--sse-customer-key` option (EnterpriseDB/barman#973). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Florian Schauer <florian@schauer.to>
The barman-cloud commands refuse --sse-customer-key together with --encryption, so a configuration setting both fails on the first backup. Reject it at validation time instead, on the data and wal encryption fields. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Andrei Nistor <andrei@nistor.tech>
A single process can run barman-cloud commands for different object stores at the same time. A replica cluster, for instance, archives WAL to its own object store while restoring it from the source one. With one shared key file, each command overwrote or removed the key of the other: a missing file made the command fail, and a different key made barman-cloud upload objects encrypted with the key of another object store, which the configured key can no longer decrypt. Materialize each key in a file whose path depends on the referenced secret and key, and point the barman-cloud commands at that same path. Signed-off-by: Armando Ruocco <armando.ruocco@enterprisedb.com>
The object stores sharing a key secret share its file, and a replica cluster can archive and restore WAL at the same time. WriteFileAtomic names its temporary file after the current second, so two concurrent writes of the same key could use the same temporary file: one of them could then fail its rename, or truncate the key file the other had just moved into place, making barman-cloud read an invalid key. Signed-off-by: Armando Ruocco <armando.ruocco@enterprisedb.com>
The barman-cloud-check-wal-archive command is the only one without the --sse-customer-key option, and it rejects it. Since the check runs before the first WAL file is archived, an object store with an SSE-C key could never start archiving. The command only lists the objects in the bucket, so it does not need the key. Signed-off-by: Armando Ruocco <armando.ruocco@enterprisedb.com>
The declaration of the mutex had split it from the function, so it documented the mutex instead. Signed-off-by: Armando Ruocco <armando.ruocco@enterprisedb.com>
| // When set, every object barman-cloud uploads to and downloads from | ||
| // S3 is encrypted with this key using the AWS SSE-C protocol | ||
| // (the `--sse-customer-key` barman-cloud option). |
There was a problem hiding this comment.
I think this reads better, right? The previous statement was kind of saying that objects are encrypted during download.
| // When set, every object barman-cloud uploads to and downloads from | |
| // S3 is encrypted with this key using the AWS SSE-C protocol | |
| // (the `--sse-customer-key` barman-cloud option). | |
| // When set, it is the key barman-cloud uses to encrypt objects uploaded | |
| // to S3 and to decrypt them when they are read back, following the AWS | |
| // SSE-C protocol (the `--sse-customer-key` barman-cloud option). |
|
@schaurian @armru I also tested this PR locally and everything went fine. I would just like to ask if you guys see any security concern in not deleting the secret key files. As I understand, if sseCustomerKey is removed, points to another Secret, or the Secret is deleted, the old file remains there forever. |
Summary
Adds support for S3 Server-Side Encryption with Customer-provided keys (SSE-C) to the barman-cloud library.
A new optional
sseCustomerKeyfield onS3Credentialsreferences a secret holding a base64-encoded 256-bit AES key. When set, the key is materialized to a file and passed to every barman-cloud command via the--sse-customer-key file://option.This is the library-side change needed to close cloudnative-pg/plugin-barman-cloud#646 — S3-compatible providers such as Hetzner Object Storage only support SSE-C for encryption at rest, so the existing bucket-managed
encryptionfield (SSE-S3 / SSE-KMS) cannot be used with them.What changed
pkg/api/config.goSSECustomerKey *SecretKeySelectorfield onS3Credentials(+ generated deepcopy)pkg/utils/constants.goSSECustomerKeyFileLocation— the on-disk path for the materialized keypkg/credentials/env.gopkg/command/commandbuilder.go--sse-customer-key file://…in the sharedappendCloudProviderOptionspkg/command/commandbuilder_test.gopkg/api/webhooks/config.gosseCustomerKeycombined withdata.encryption/wal.encryption(by @andrein, with tests)Design notes
appendCloudProviderOptions, through which every command builder already routes (barman-cloud-backup,-wal-archive,-wal-restore,-restore,-backup-list,-backup-delete,-check-wal-archive). SSE-C requires the key on every read and write, so covering the shared chokepoint avoids the classic "backups succeed but restores fail because a read path was missed" footgun.reconcileGoogleCredentials→ a/controller/...file written atomically with0600), so no new volume-mount plumbing is needed in consumers; the same code path serves both the operator and the plugin sidecar.encryption. barman-cloud rejects--sse-customer-keytogether with--encryptionat run time, soValidateBackupConfigurationnow reports one error per offending field (data.encryption,wal.encryption) whensseCustomerKeyis set, instead of letting the first backup fail.envSetAWSCredentials, so it works with explicit keys, session tokens, andinheritFromIAMRole.Dependency / rollout
Runtime use requires Barman ≥ 3.20.0, which ships the
--sse-customer-keyoption (EnterpriseDB/barman#973, released 2026-08-27). Theplugin-barman-cloudsidecar pins Barman 3.20.0 since v0.15.0. The plugin side of this feature is cloudnative-pg/plugin-barman-cloud#1017.Testing
task cirun locally (commitlint, spellcheck, golangci-lint v2.13.2,go test ./..., uncommitted-drift check): all green.backup-list, retention viabackup-delete, and recovery into a newClusterall working withEncryption: SSE-Cobjects) — see the comments below.🤖 Generated with Claude Code