Skip to content

feat: Make secret volumes read-only for the Pod's fsGroup - #776

Draft
lfrancke wants to merge 1 commit into
fix/write-secret-files-nofollowfrom
feat/secret-volume-read-only-group
Draft

lfrancke wants to merge 1 commit into
fix/write-secret-files-nofollowfrom
feat/secret-volume-read-only-group

Conversation

@lfrancke

@lfrancke lfrancke commented Nov 11, 2026 •

Copy link
Copy Markdown
Member

Description

With fsGroupPolicy: File, the kubelet gives the Pod's fsGroup read and write access to everything in a secret volume after each publish (files 0660, directories g+rwx with setgid).
Products only need to read their secrets, and secret-operator writes into the volume again on a repeated NodePublishVolume, so the Pod should not be able to change its entries.

The node service now advertises the VOLUME_MOUNT_GROUP capability.
The kubelet then passes the fsGroup in NodePublishVolume instead of changing ownership itself, and secret-operator sets it on every directory (0750) and file (0640) it writes.
Files of existing volumes are corrected the next time they are written, i.e. when the Pod is recreated.

The CSI spec recommends read-write access for the mount group (SHOULD). Read-only is deliberate.

Builds on #775.

Breaking: products that write into a secret volume fail.
I checked all operators, operator-rs and docker-images on main.
One place writes into a secret volume: kafka-operator with OPA TLS imports the OPA CA into /stackable/tls-kafka-internal/truststore.p12 (security.rs), and opa.authorizer.truststore.path points at that file.
That needs to be fixed in kafka-operator before this is merged.
The check was static, so the integration tests of all products should run against this before merging.

Tested on k3d (k3s v1.37.2): the volume is drwxr-x--- / -rw-r----- with the fsGroup as group, the product (uid 1000) can read its secrets, and creating or deleting entries fails with Permission denied.

Not to be merged before the 26.11 release.

Definition of Done Checklist

  • Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
  • Please make sure all these things are done and tick the boxes

Author

  • Changes are OpenShift compatible
  • CRD changes approved
  • CRD documentation for all fields, following the style guide.
  • Helm chart can be installed and deployed operator works
  • Integration tests passed (for non trivial changes)
  • Changes need to be "offline" compatible
  • Links to generated (nightly) docs added
  • Release note snippet added

Reviewer

  • Code contains useful comments
  • Code contains useful logging statements
  • (Integration-)Test cases added
  • Documentation added or updated. Follows the style guide.
  • Changelog updated
  • Cargo.toml only contains references to git tags (not specific commits or branches)

Acceptance

  • Feature Tracker has been updated
  • Proper release label has been added
  • Links to generated (nightly) docs added
  • Release note snippet added
  • Add type/deprecation label & add to the deprecation schedule
  • Add type/experimental label & add to the experimental features tracker

With fsGroupPolicy File, the kubelet gives the Pod's fsGroup read and write
access to everything in the volume after each publish. The node service now
advertises the VOLUME_MOUNT_GROUP capability, so the kubelet passes the
fsGroup instead, and the volume is owned by that group with 0750 for
directories and 0640 for files.

The CSI spec recommends read-write access for the mount group. Read-only is
deliberate: the publish path writes into the volume again on a repeated
NodePublishVolume, so the Pod must not be able to change its entries.
Products that write into a secret volume break with this change.
@lfrancke
lfrancke changed the base branch from main to fix/write-secret-files-nofollow November 11, 2026 10:56
@lfrancke lfrancke changed the title fix: Write secret files without following symlinks feat: Make secret volumes read-only for the Pod's fsGroup Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant