[Cosmos] Backport PPCB fixes to 4.71.3-hotfix - #50279
Draft
Abhijeet Mohanty (jeet1995) wants to merge 5 commits into
Draft
Conversation
* Fix PPCB failback with missing or stale addresses --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Improve PPCB failback diagnostics * Validate PPCB state in diagnostics E2E tests * Limit PPCB diagnostics assertion to data-plane requests * Log PPCB failback backlog progress * Add PPCB failback remaining meter * Clarify PPCB failback meter name * Reduce PPCB failback meter allocations * Track PPCB pending recoveries by collection * Correlate PPCB failback recovery diagnostics * Harden PPCB failback recovery tests * Fix PPCB all-region diagnostics assertion * Optimize PPCB diagnostics snapshots * Cache PPCB diagnostics snapshots * Align PPCB failback flow with main * Add benchmark fault injection support. * Harden PPCB failback telemetry Use a single injectable logger, prevent backlog telemetry failures from escaping the recovery flow, and align the pending failback metric name. * Scope PR to PPCB diagnostics Remove benchmark, fault-injection, metric, and recovery behavior changes. Retain immutable CosmosDiagnostics PPCB snapshots, lifecycle E2E assertions, and WARN logging for every failback failure. * Reduce PPCB diagnostics overhead Reuse immutable PPCB map references in response snapshots and shorten per-region diagnostic field names. * Refactoring * Simplify PPCB diagnostics snapshots Represent holder state with one volatile immutable-map reference and align the compact timestamp serialization test. * Ignore updates to empty PPCB diagnostics Make updates to the shared uninitialized diagnostics sentinel a no-op and verify it remains null-serializing. * Avoid copying PPCB diagnostics state Retain the live PPCB diagnostics map reference to avoid per-publication map and wrapper allocations, accepting weak consistency. * Log PPCB diagnostics lifecycle snapshots Emit one full CosmosDiagnostics JSON payload for failed, post-failover, and post-failback E2E phases for PR evidence. * Document PPCB diagnostics improvements Add the unreleased changelog entry for per-region PPCB snapshots and failback WARN logging. * Add PPCB failback outcome diagnostics Track the latest background failback attempt time, outcome, and failure reason per partition-region; validate lifecycle state in E2E and focused recovery tests. * Reset unavailable timestamp after PPCB failback Use the available-state sentinel when recovery moves a region to HealthyTentative and cover it in the scheduled recovery test. * Refine PPCB failback diagnostics Keep failback attempt metadata partition-scoped, retain only the latest full failure message per region, and clear retained messages when recovery backlog drains.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 16ce0941-555c-4190-8c4d-96c705086350
|
Azure Pipelines: 35 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Member
Author
|
/azp run java - cosmos - tests |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
Member
Author
|
/azp run java - cosmos - ci |
|
Azure Pipelines: 1 pipeline(s) were filtered out due to trigger conditions. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Backports the PPCB fixes into Azure Cosmos 4.71.3-hotfix:
partitionLevelCircuitBreakerCfgin diagnostics client configuration (PR Fix partitionLevelCircuitBreakerCfg missing from CosmosDiagnostics clientCfgs #49734)Comparison baseline
This draft targets$(System.Collections.Specialized.OrderedDictionary.base), pinned to Azure Cosmos 4.71.2-hotfix at $ (System.Collections.Specialized.OrderedDictionary.baselineSha).
The hotfix head is owned by
jeet1995/azure-sdk-for-java.Validation
azure-cosmos,azure-cosmos-test, andazure-cosmos-testsbuild successfully across all four hotfix worktrees.Source PRs: #50182, #50158, #49734.