From 2da2c83d25d20bb0005eea3a37e6839cd7f81ce5 Mon Sep 17 00:00:00 2001 From: Daniel Garcia Sardina Date: Mon, 5 Oct 2026 09:22:19 +0300 Subject: [PATCH] feat(backup): allow zstd compression for base backups Upgrade the barman-cloud library to a version that includes cloudnative-pg/barman-cloud#310, which adds `zstd` to the `DataBackupConfiguration.compression` enum. Regenerate the ObjectStore CRD and the release manifest, and update the compression docs so that `zstd` is no longer listed as WAL-only. Barman 3.20.0 added `--zstd` to `barman-cloud-backup`; the sidecar image already ships barman 3.20.1 with the `zstandard` extra, and `barman-cloud-restore` picks the decompressor from the `.tar.zst` suffix recorded in `backup.info`, so no plugin code changes are needed. The library bump also pulls in `s3AddressingStyle` (cloudnative-pg/barman-cloud#288), which therefore appears in the CRD. Closes #1104 Signed-off-by: Daniel Garcia Sardina --- .../bases/barmancloud.cnpg.io_objectstores.yaml | 17 ++++++++++++++++- go.mod | 2 +- go.sum | 4 ++-- manifest.yaml | 17 ++++++++++++++++- web/docs/compression.md | 4 ++-- 5 files changed, 37 insertions(+), 7 deletions(-) diff --git a/config/crd/bases/barmancloud.cnpg.io_objectstores.yaml b/config/crd/bases/barmancloud.cnpg.io_objectstores.yaml index bb7db1e6e..da95cfd59 100644 --- a/config/crd/bases/barmancloud.cnpg.io_objectstores.yaml +++ b/config/crd/bases/barmancloud.cnpg.io_objectstores.yaml @@ -144,12 +144,13 @@ spec: description: |- Compress a backup file (a tar file per tablespace) while streaming it to the object store. Available options are empty string (no - compression, default), `gzip`, `bzip2`, `lz4`, and `snappy`. + compression, default), `gzip`, `bzip2`, `lz4`, `snappy`, and `zstd`. enum: - bzip2 - gzip - lz4 - snappy + - zstd type: string encryption: description: |- @@ -255,6 +256,16 @@ spec: HistoryTags is a list of key value pairs that will be passed to the Barman --history-tags option. type: object + s3AddressingStyle: + description: |- + How bucket names appear in S3 requests: `virtual` (bucket.endpoint/key), + `path` (endpoint/bucket/key) or `auto` (boto3 decides, same as unset). + Requires s3Credentials and Barman 3.18 or later. + enum: + - auto + - virtual + - path + type: string s3Credentials: description: The credentials to use to upload data to S3 properties: @@ -412,6 +423,10 @@ spec: message: use the 'serverName' plugin parameter in the Cluster resource reason: FieldValueForbidden rule: '!has(self.serverName)' + - fieldPath: .s3AddressingStyle + message: s3AddressingStyle requires s3Credentials + reason: FieldValueForbidden + rule: '!has(self.s3AddressingStyle) || has(self.s3Credentials)' instanceSidecarConfiguration: description: The configuration for the sidecar that runs in the instance pods diff --git a/go.mod b/go.mod index c86dd4bf9..29b545988 100644 --- a/go.mod +++ b/go.mod @@ -5,7 +5,7 @@ go 1.26.6 require ( github.com/cert-manager/cert-manager v1.21.2 github.com/cloudnative-pg/api v1.30.0 - github.com/cloudnative-pg/barman-cloud v0.6.0 + github.com/cloudnative-pg/barman-cloud v0.6.1-0.20261002081136-603c4849d2d9 github.com/cloudnative-pg/cloudnative-pg v1.30.1 github.com/cloudnative-pg/cnpg-i v0.6.0 github.com/cloudnative-pg/cnpg-i-machinery v0.4.2 diff --git a/go.sum b/go.sum index 7fbba1cd6..9ee6adef9 100644 --- a/go.sum +++ b/go.sum @@ -20,8 +20,8 @@ github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UF github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/cloudnative-pg/api v1.30.0 h1:L8hnvV/tPEQA1xYEi41FUBFA7FUNVGju8+SlgFlDDjI= github.com/cloudnative-pg/api v1.30.0/go.mod h1:XrKBbOWObL33si0FNuwX4uHNf5JShiZyOUqd6LxbJQo= -github.com/cloudnative-pg/barman-cloud v0.6.0 h1:OtBFmCDyVUAcgFa++FIoCCJwPfd5TtqK3PH6DGPcpkA= -github.com/cloudnative-pg/barman-cloud v0.6.0/go.mod h1:eqSPRGz/s8M0Mea8mkqiVQUTkoSquAhvJF49feh+Ks4= +github.com/cloudnative-pg/barman-cloud v0.6.1-0.20261002081136-603c4849d2d9 h1:ea1kEPU7hXyEOpGEluPJ5RAn3FuMnnnY3fzi1ms1HKQ= +github.com/cloudnative-pg/barman-cloud v0.6.1-0.20261002081136-603c4849d2d9/go.mod h1:EqfZ2L7YLkLdvoCuy90+HFzUe260qn2jXQnGz+jBKoE= github.com/cloudnative-pg/cloudnative-pg v1.30.1 h1:d1jxp0jQi7/3zrEhsv9ycD+G5ssmkNvDvTmogaGccCw= github.com/cloudnative-pg/cloudnative-pg v1.30.1/go.mod h1:k931EKEiGsGwHid+Lwo3vt4ZvnmnaIUYS0SJ29Bpivw= github.com/cloudnative-pg/cnpg-i v0.6.0 h1:LA//DLkFOLIjU0ASOpFkydZhGir9IAIDfgSsTTX9IpU= diff --git a/manifest.yaml b/manifest.yaml index 118beaf42..e30256e4e 100644 --- a/manifest.yaml +++ b/manifest.yaml @@ -143,12 +143,13 @@ spec: description: |- Compress a backup file (a tar file per tablespace) while streaming it to the object store. Available options are empty string (no - compression, default), `gzip`, `bzip2`, `lz4`, and `snappy`. + compression, default), `gzip`, `bzip2`, `lz4`, `snappy`, and `zstd`. enum: - bzip2 - gzip - lz4 - snappy + - zstd type: string encryption: description: |- @@ -254,6 +255,16 @@ spec: HistoryTags is a list of key value pairs that will be passed to the Barman --history-tags option. type: object + s3AddressingStyle: + description: |- + How bucket names appear in S3 requests: `virtual` (bucket.endpoint/key), + `path` (endpoint/bucket/key) or `auto` (boto3 decides, same as unset). + Requires s3Credentials and Barman 3.18 or later. + enum: + - auto + - virtual + - path + type: string s3Credentials: description: The credentials to use to upload data to S3 properties: @@ -411,6 +422,10 @@ spec: message: use the 'serverName' plugin parameter in the Cluster resource reason: FieldValueForbidden rule: '!has(self.serverName)' + - fieldPath: .s3AddressingStyle + message: s3AddressingStyle requires s3Credentials + reason: FieldValueForbidden + rule: '!has(self.s3AddressingStyle) || has(self.s3Credentials)' instanceSidecarConfiguration: description: The configuration for the sidecar that runs in the instance pods diff --git a/web/docs/compression.md b/web/docs/compression.md index cfc6355dc..2b664accd 100644 --- a/web/docs/compression.md +++ b/web/docs/compression.md @@ -18,7 +18,7 @@ for space, speed, or a balance of both. - `lz4` - `snappy` - `xz` (WAL only) -- `zstd` (WAL only) +- `zstd` Compression settings for base backups and WAL archives are configured independently. For implementation details, refer to the corresponding API @@ -42,4 +42,4 @@ network throughput. | gzip | 116,281 | 3,077 | 395 | 91 | 4.3:1 | | snappy | 8,134 | 8,341 | 395 | 166 | 2.4:1 | -Numbers come from a 2021 Barman proof of concept ([EnterpriseDB/barman#344](https://github.com/EnterpriseDB/barman/issues/344#issuecomment-992547396)), which predates `lz4` support for base backups. `lz4` is not yet benchmarked. +Numbers come from a 2021 Barman proof of concept ([EnterpriseDB/barman#344](https://github.com/EnterpriseDB/barman/issues/344#issuecomment-992547396)), which predates `lz4` and `zstd` support for base backups. `lz4` and `zstd` are not yet benchmarked.