export-tar: support sparse files (GNU sparse format 1.0)

[ThomasWaldmann]

Fixes #2562.

Add a --sparse option to export-tar (BORG and PAX tar formats only): files whose content contains runs of all-zero chunks are written as sparse tar members in GNU sparse format 1.0 (what GNU tar creates in POSIX mode), storing only a hole map and the non-hole data.

How

The issue's original blocker was that the sparse map must precede the file data in the tar stream. That is solved by computing the hole map from the item's chunk list metadata alone, before fetching any data: an all-zero chunk is detected by comparing its id against the (memoized) id of an all-zero chunk of the same size, reusing the zeros-shortcut heuristic of DownloadPipeline.fetch_many (extracted into a shared zero_chunk_flags()). The memo is additionally warmed up for power-of-two chunk sizes and each file's last chunk size, so single (non-repeated) zero chunks — the only chunk of an all-zero file, or a trailing hole's tail — are detected as well. The zero chunks of a sparse member are never fetched from the repository at all.

Python's tarfile reads GNU sparse members (import-tar has always consumed such tarballs correctly) but cannot write them, so the writing side crafts the pax records and the map itself and streams map + data segments through tarfile.addfile():

• pax record order matters for in-order readers: path (mangled GNUSparseFile.0/<name>, like GNU tar) first, GNU.sparse.name / GNU.sparse.realsize last, so the real name/size win.
• no pax size record may be written for a sparse member (readers would desync when recalculating the next-header offset from it); a stored (non-hole) size beyond the 12-digit octal ustar field limit (8 GiB) is base-256 encoded into that field instead (the GNU/star big-number encoding, accepted by GNU tar, libarchive and python's tarfile) — so both stored data and logical size (GNU.sparse.realsize) are effectively unlimited.
• members where the map would cost more than the holes save also stay dense; without --sparse, output is byte-identical to before.
• --sparse with --tar-format=GNU errors out (old-GNU sparse headers are not PAX-based and deprecated).

Hole detection is chunk-granular (like extract --sparse) and does not depend on the original file having been sparse on disk.

Interop

Verified against GNU tar 1.35 (byte-level comparison of the member layout gtar produces, including the terminal (realsize, 0) map entry convention), bsdtar/libarchive 3.7.4, python tarfile in both r: and r| modes, and borg import-tar round trips (PAX and BORG formats).

Tests

Unit tests for the map builders and zero_chunk_flags; integration tests parametrized over a fixed chunker and a small-target fastcdc chunker (16 KiB target / 64 KiB max, so the small test files' sparse ranges yield multiple pure repeated all-zero chunks, as with the default chunker on real files); round trips (PAX/BORG × both chunkers), GNU tar extraction interop (incl. on-disk sparseness check on Linux), hardlinks, --strip-components, all-hole/trailing-hole/empty/unaligned edge cases, not-worthwhile fallback, and the GNU-format error.

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 95.52239% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.94%. Comparing base (91b030e) to head (9547a53).
⚠️ Report is 33 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
src/borg/archiver/tar_cmds.py	95.12%	2 Missing and 4 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #10128      +/-   ##
==========================================
- Coverage   87.05%   86.94%   -0.11%     
==========================================
  Files         101      101              
  Lines       17848    17984     +136     
  Branches     2705     2737      +32     
==========================================
+ Hits        15537    15637     +100     
- Misses       1609     1638      +29     
- Partials      702      709       +7     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.