DOC-2292/DOC-2015: Document shadow link role synchronization (26.2)#1780
Merged
Conversation
Documents the new v26.2 Schema Registry API-mode replication (shadow_schema_registry_api) for shadow links: - New page: Migrate Schemas from Confluent Schema Registry (intro, use cases, prerequisites, limitations, config steps, filtering, context mapping, validation policy, monitoring, failover) - setup.adoc: present topic mode vs API mode in the Schema Registry synchronization section - overview.adoc: add Schema Registry Sync task tab and update the replication lists - monitor.adoc: add Schema Registry sync status bullet - nav.adoc: add nav entry under Shadowing Feature is unreleased (ENG-1171, targets 26.2). DRAFT-VERIFY comments mark items to re-check after redpanda#30999 and redpanda#30984 merge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Intro now follows what-it-does / how-it-works / benefit / when-to-use instead of opening with an imperative. Duplicate glossterms downgraded now that first mentions moved into the intro. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lication Full-file example anchors the per-setting fragments that follow: name + client_options + topic sync (from the canonical setup.adoc sample) plus the complete shadow_schema_registry_api section (field names verified against rpk types_test.go). Links to Configure Shadowing for the full connection/auth reference. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds lines=20-38 highlighting to the sample shadow link configuration so the shadow_schema_registry_api block stands out from the framing sections, and points to it from the lead-in. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Documents the Roles Migrator task (ENG-1233 rpk + ENG-898 core): - setup.adoc: role_sync_options in the sample config file + new Role filtering section (empty-default gotcha, full-mirror semantics, credentials deliberately excluded, ACL sync required for end-to-end authorization, permissions/network notes) - overview.adoc: Roles Migrator task tab + RBAC roles in the replication list - rbac.adoc: cross-link TIP to role filtering All content gated ifndef::env-cloud (rpk rejects role_sync_options for Cloud in 26.2). Field names verified against rpk types.go and PR #30909; behavior from roles_migrator.cc and ducktape e2e tests. DRAFT-VERIFY comments mark SME questions for Andrew Nguyen and Rogger Vasquez. Stacked on the DOC-2254 branch to avoid conflicts in shared files. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Addresses review feedback asking for explicit guidance on choosing between the two Schema Registry sync modes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
nguyen-andrew
left a comment
nguyen-andrew
left a comment
Member
There was a problem hiding this comment.
Looks good, just added some responses to questions
- Drop the "security vulnerability" rationale for excluding SCRAM credentials; state only that Shadowing does not replicate them - Document that failover acts on topic data only: sync tasks keep running until paused per-task or the link is deleted - Document Apache Kafka source behavior: Roles Migrator enters LINK_UNAVAILABLE without affecting other tasks - Document role sync counts in the rpk shadow status Reason field (latest cycle only), in setup.adoc and the monitor.adoc status list - Remove the DRAFT-VERIFY comment block: all four questions answered in review Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…as-confluent.adoc Co-authored-by: Gellért Peresztegi-Nagy <gellert.nagy@redpanda.com>
Corrections and additions from pgellert's review: - Two cluster tunables do exist (schema_registry_sync_memory_bytes, schema_registry_sync_parallelism); replace the "no cluster configuration properties" claim - mTLS is supported alongside HTTP basic auth for the source registry - Prerequisites: destination contexts owned by the link must be empty (rest of the registry need not be); qualified subjects config required for non-default contexts - Limitations: topic data and schema replication are not coordinated (SerDes records can reference not-yet-replicated schema IDs) - Remove topic_metadata_sync_options from the sample config and fix the highlight range - Filters accept literal names only (no wildcards/prefixes) - exact context mappings must be exhaustive: unmapped source contexts fail the task; recommend scoping source_filter - List the unsupported Confluent features (rule sets, metadata tags, override/default metadata and rule sets, compatibility groups) - Monitoring: task runs on the _schemas partition leader; counters are ephemeral and reset on restart or leadership movement - Failover: pausing the SR sync task (paused: true) is a required step; failover alone does not stop schema replication - overview.adoc: only API mode runs as a separate task in link status Resolves all DRAFT-VERIFY items except the sync-defaults re-check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
default_tail_interval=10s, default_full_sync_interval=5m, default_max_source_requests_per_second=30 confirmed in src/v/cluster_link/model/types.h at the v26.2.1-rc3 tag, matching the documented values. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Trevor Blackford <trevor.blackford@redpanda.com>
…as-confluent.adoc Co-authored-by: Gellért Peresztegi-Nagy <gellert.nagy@redpanda.com>
Per Gellert's answers to Trevor's review questions: - Rework the Select contexts and subjects example so the two lists no longer overlap (the old example's subject was already inside a listed context), and add the union semantics from shadow_link.proto: contexts replicate in full, subjects use qualified syntax, both lists union, overlaps replicate once - Extend the SerDes coordination limitation with the user impact: no replication errors (schema IDs not validated during topic data replication), but consumers fail on unreplicated schema IDs, so wait for schema replication to catch up before consuming Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
cloud-docs has no consuming stub for the new page (the cloud twin is deferred to DOC-2307), so xrefs to it from the shared regions of the single-sourced shadowing pages would break every cloud build with target-not-found errors. Gate the cross-link sentences ifndef::env-cloud, gate monitor.adoc's Schema Registry sync status bullet entirely (it describes API-mode-only output), and give the overview's replication paragraph a cloud variant that mentions topic-mode replication only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
10 tasks
Contributor
Author
|
@copilot resolve the merge conflicts in this pull request |
Contributor
Resolved in the latest commit. The only conflict was in |
Base automatically changed from
DOC-2254-confluent-schema-registry-migration
to
beta
July 20, 2026 23:29
…erged # Conflicts: # modules/manage/pages/disaster-recovery/shadowing/monitor.adoc # modules/manage/pages/disaster-recovery/shadowing/overview.adoc
✅ Deploy Preview for redpanda-docs-preview ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Feediver1
commented
Jul 21, 2026
Feediver1
left a comment
Feediver1
left a comment
Contributor
Author
There was a problem hiding this comment.
Final-pass review: shadow link role synchronization (#1780)
Files reviewed: 4 (setup role-filtering section, overview Roles Migrator tab + RBAC bullet, monitor role-counts bullet, rbac.adoc TIP)
Overall assessment: Content-complete and SME-approved. The squash-merge aftermath from #1776 is now fully cleaned up: the PR diff is back to its true role-sync-only payload (+79/−1 across 4 files) and mergeability is restored.
What happened around the #1776 merge (for the record)
- #1776 squash-merged to
beta(5ef271e) and this PR retargeted tobetaas planned. - Because the squash left this stacked branch's copy of the DOC-2254 commits outside beta's history, the PR diff temporarily showed all of #1776's content (+437 across 6 files) and reported CONFLICTING.
- Copilot's conflict-resolution commits were verified correct — its monitor.adoc resolution (combining the SR-sync-status and role-sync bullets in one
ifndef::env-cloud[]block) is exactly the right treatment for the collision between #1776's cloud-gating and this PR's addition. - The final cleanup (4389e9b) merged
betainto the branch, resolving three same-shaped conflicts (role-sync additions vs. absent-on-beta — keep-branch in all cases, verified no content lost). The diff now collapses to the reviewed role-sync content, and beta's latest (including #1776's final gating and xref cleanup) is incorporated.
Review state
- Approved by Andrew Nguyen (2026-07-16) after all four of his DRAFT-VERIFY answers were implemented (6f6e95a, 78a3039): failover semantics, Apache Kafka source behavior (
LINK_UNAVAILABLE), status counters in theReasonfield, and the 26.2 version requirement. His two threads are content-addressed with commit-cited replies — resolve-clicks only. - No DRAFT-VERIFY markers remain in the role-sync content.
Deferred by design (tracked elsewhere)
- Cloud twins DOC-1814/DOC-2306 own the future un-gating when Cloud supports
role_sync_options. - The "activates after the upgrade completes on all brokers" phrasing should acknowledge deferred finalization once DOC-2013's page lands (noted on that ticket).
- rpk shadow reference pages regenerate at 26.2 GA (config template + describe/status gain Role Sync sections).
Verdict
Ready to merge once checks pass (state is UNSTABLE = checks running post-push). This closes DOC-2292 and DOC-2015.
🤖 Generated with Claude Code
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Documents shadow link role synchronization shipping in v26.2: the new Roles Migrator task that continuously replicates RBAC role definitions and memberships from source to shadow cluster (ENG-1233 rpk + ENG-898 core). All engineering PRs are merged to dev, including the
shadow_link_role_syncfeature flag (redpanda#30976, 2026-07-08).Resolves https://redpandadata.atlassian.net/browse/DOC-2292
Resolves https://redpandadata.atlassian.net/browse/DOC-2015
Review deadline:
Changes
role_sync_optionsadded to the sample config file; new Role filtering section with:describecovers rolesAll new content is
ifndef::env-cloud[]-gated: rpk explicitly rejectsrole_sync_optionsfor Cloud shadow links in 26.2. Cloud twins (DOC-1814/DOC-2306) pick this up when Cloud support lands.Sources of truth
Field names from
src/go/rpk/pkg/cli/shadow/types.go+ PR #30909's sample config; behavior fromroles_migrator.cc,role_reconcile.h, andshadow_link_role_sync_test.py. Engineering PRs: redpanda#30838, #30932, #30946, #30909, #30978, #30976, #30731. Full research report on DOC-2292.✅ Verified in review (answers from @nguyen-andrew, applied in 6f6e95a)
LINK_UNAVAILABLEwithout affecting other tasks (now documented)rpk shadow statusReason field, latest cycle only, not Prometheus (now documented in setup.adoc + monitor.adoc)Also per review: removed the "security vulnerability" rationale for excluding SCRAM credentials.
Related
Page previews
/streaming/26.2/manage/disaster-recovery/shadowing/setup/#role-filtering/streaming/26.2/manage/disaster-recovery/shadowing/overview/#shadow-link-tasks/streaming/26.2/manage/security/authorization/rbac/Checks
🤖 Generated with Claude Code