feat: --drop-unresolvable-principals flag + drop-aware connection scaffolding (1/4)

[michael-richey]

Context

Stacked PR 1 of 4. Base: main.

Exported resources can reference roles/principals that were deleted from the source before their state was captured, leaving references that can never resolve. Today one such dead reference makes connect_resources() raise ResourceConnectionError and skip the entire resource's access-control sync.

This series adds an opt-in --drop-unresolvable-principals flag: drop a principal absent from both destination and source (permanently gone), keep syncing the rest, and treat an emptied binding/list as a normal resource-connection failure. Default-off = byte-for-byte unchanged behavior.

Interaction with --skip-failed-resource-connections

Restriction policies are resource connections, so the existing flag remains authoritative:

• Without --skip-failed-resource-connections, an emptied binding/list raises ResourceConnectionError and the resource is skipped.
• With --skip-failed-resource-connections=true, that connection failure is intentionally suppressed and the resource continues syncing, even though the destination resource may become unrestricted.
• The permitted escalation is explicit: an immediate ERROR says DESTINATION RESOURCE MAY BE UNRESTRICTED, the successful action metric carries risk:empty_restriction_policy, and the end-of-run ERROR summary lists resources that actually synced through this override.

This PR (scaffolding — inert when the flag is off)

• --drop-unresolvable-principals CLI option (in _diffs_options → sync/diffs/migrate), threaded through Configuration.
• BaseResource._resolve_or_drop: three-way resolver (destination-present / source-present "not yet synced" → hard-fail as today / absent-from-both → droppable only under the flag). For --minimize-reads, it rechecks destination after ensure_resource_loaded() before classifying the reference as pending or stale, because lazy loading can populate the destination mapping.
• Shared _filter_stale_binding_principals, _filter_stale_flat_roles, _raise_connection_error_if_any helpers. Lists are rebuilt (never index-mutated) to avoid the enumerate/index-shift skip bug.
• ResourceConnectionResult gives the base method and model overrides a named, type-safe result contract; successful suppressed-risk syncs set empty_binding_escalation explicitly rather than relying on truthiness.
• ResourceConnectionError.empty_binding_risk; separate Counter buckets for skipped risks and successful suppressed-risk syncs; config.counter back-reference; truthful end-of-run summaries.
• _apply_resource_cb tags both skipped and successful suppressed-risk metrics with risk:empty_restriction_policy.

No model opts into the drop-aware helpers yet — runtime behavior remains unchanged until PRs 2–4.

Testing

New unit tests cover the named connection-result contract, resolver branches—including lazy loads that populate destination-only and both source/destination—filtering, counters/reset, exception metadata, both empty-binding connection modes, truthful ERROR logs, skipped/success metrics, end-of-run summaries, and CLI flag round-trip. Focused tests and ruff are green locally; CI is pending the updated branch.

[Copilot]

Pull request overview

Adds opt-in scaffolding for handling unresolvable principals/roles during resource connection, including new counters/summary output and metric tagging for the “empty restriction policy” risk path, while keeping default behavior unchanged when the flag is off.

Changes:

• Introduces --drop-unresolvable-principals CLI option and threads it through Configuration.
• Adds drop-aware connection helpers to BaseResource plus new Counter buckets and apply-summary reporting for dropped principals and empty-binding risk/escalation.
• Extends ResourceConnectionError with empty_binding_risk metadata and tags both skipped and successful suppressed-risk metrics.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/unit/test_workers.py	Adds unit coverage for new Counter buckets and reset behavior.
tests/unit/test_resource_utils.py	Tests new ResourceConnectionError.empty_binding_risk behavior.
tests/unit/test_drop_unresolvable_principals_cli.py	Verifies CLI option is accepted by sync/diffs/migrate and rejected by import.
tests/unit/test_base_resource.py	Adds tests for _resolve_or_drop and empty-binding risk behavior.
tests/unit/test_apply_summary_handler.py	Adds coverage for new apply-summary warning/error lines.
tests/unit/test_apply_resource_connection_error.py	Validates risk tagging and counter recording for skipped and successful escalation paths.
datadog_sync/utils/workers.py	Adds new Counter buckets + sets config.counter back-reference.
datadog_sync/utils/resources_handler.py	Tags metrics and records counters for empty-binding risk/escalation; updates summary emission.
datadog_sync/utils/resource_utils.py	Adds empty_binding_risk to ResourceConnectionError.
datadog_sync/utils/configuration.py	Threads drop_unresolvable_principals through config + adds optional counter reference.
datadog_sync/utils/base_resource.py	Adds shared resolver/filter helpers and shared empty-binding risk raising behavior.
datadog_sync/commands/shared/options.py	Adds the --drop-unresolvable-principals click option to _diffs_options.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.