feat(transfer): align cross-alias copy, move, and diff on source identity

[loverustfs]

Related work

Builds on #351 (mirror incremental identity) and #353 (cross-alias cp). Both are merged into this branch, so review this after those two land; the diff against main includes them.

Also resolves the mv half of the cross-alias gap and the diff comparison inconsistency.

Background

#351 and #353 are each correct on their own, but together they leave a seam. A remote-to-remote transfer that streams through the client cannot preserve the source ETag — the destination computes its own, and multipart completion makes it differ even when the bytes are identical. #351 solved this for mirror by recording the source ETag in x-amz-meta-rc-source-etag. #353 added cross-alias cp without writing that key.

The result breaks the most common migration shape:

rc cp --recursive stage/data/ prod/archive/    # first full pass
rc mirror stage/data/ prod/archive/ --compare auto   # incremental

The second command re-copies the entire tree, because it cannot tell a faithful cross-alias copy from a changed object.

Two related asymmetries came out of the same audit:

• rc mv still rejected cross-alias remote moves with unsupported_feature even though rc cp now supports them, so the two commands disagreed about what is possible. Its S3-to-S3 path also duplicated copy logic instead of reusing cp, unlike its local↔remote paths.
• rc diff compared only size and the listed ETag, so it reported ≠ for exactly the pairs mirror --compare auto calls synchronized. Using diff to verify a mirror run gave contradictory answers.

Root cause

Three commands each decided independently whether two remote objects hold the same data, and only one of them knew about the recorded source identity.

Solution

A shared object_identity module owns the metadata key, the reader, and the writer, so the three commands cannot drift apart again. The first commit is a pure refactor with no behavior change; the second changes behavior.

• cp: cross-alias uploads record the source ETag. This is rc bookkeeping rather than user data, so it is written even with --metadata-directive replace, while that flag still drops source user metadata. Same-alias copies use server-side CopyObject, which preserves the ETag, and are untouched.
• mv: cross-alias remote moves now work by reusing the cp streaming path rather than reimplementing it, and inherit its constraints (SSE-C rejected, --storage-class rejected above the multipart threshold). The source is deleted only after its own copy succeeds, so a failed or partial move never destroys data that did not arrive. Same-alias moves keep server-side CopyObject.
• diff: adds --compare auto|etag|size, matching rc mirror --compare. auto reads the recorded identity through HeadObject, and only for same-size pairs whose listed ETags differ, so an unchanged tree costs no extra requests and etag/size never issue HeadObject.

Behavior changes

• rc diff default moves from the old size-plus-ETag rule to auto. A target recording the source ETag is now Same instead of Different, so the command exits 0 where it previously exited 1. Pass --compare etag to keep the previous comparison.
• rc diff no longer treats entries with unknown size on either side as equal; they are reported as different.
• rc mv between different aliases succeeds instead of returning the unsupported-feature exit code. Automation relying on that rejection to detect an unsupported operation must be updated.

This PR must be marked BREAKING because docs/reference/rc/cp.md, mirror.md, mv.md, and diff.md are protected CLI behavior contracts. No JSON schema or config schema_version bump applies.

Tests

• Shared helper: read/write round trip, x-amz-meta- prefix and case variants, empty values rejected, unrelated metadata preserved
• cp: identity recorded, recorded under --metadata-directive replace, omitted when the source has no ETag
• diff: each compare mode, mismatched identity, size mismatch and unknown size in all three modes, HeadObject lookup gating, --diff-only
• Integration: cross-alias cp upload carries the metadata header; cross-alias mv does LIST/GET/PUT/DELETE and never sends x-amz-copy-source; a failed move takes the cross-alias path and leaves the source intact; diff --compare auto reports same via HeadObject while etag reports different with no HeadObject
• End-to-end: cp --recursive across aliases, then mirror --compare auto skips instead of recopying — the exact gap this PR closes, with a mock that persists what the upload wrote
• Regression: scripts/regression/cross-alias-identity.sh

Every new assertion was verified against a deliberately mutated implementation (identity write disabled, cross-alias dispatch disabled, identity comparison disabled) to confirm it fails when the behavior regresses rather than passing unconditionally.

Validation

cargo fmt --all --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
./scripts/regression/cross-alias-identity.sh
./scripts/regression/mirror-identity.sh
./scripts/regression/cross-alias-copy.sh

All 59 test binaries pass with zero failures, and the two prerequisite regression suites still pass unchanged.

  

[cxymds]

Review note: this branch includes the identity race and cross-alias source-consistency issues identified in PRs #351 and #353. The P1 move deletion issue below makes the source race destructive.

[cxymds]

Addressed in a2133b5. In addition to the inline fixes, this branch now includes the inherited mirror source preflight and cross-alias post-download revalidation. Move deletion is version/ETag conditional, and diff detects LIST/HEAD snapshot conflicts. Workspace checks pass.