fix: decouple client download from merge; stop deleting parts under in-flight readers

[peter-svensson]

Problem

Two issues in Storage.download() for unmerged entries (large caches that haven't been merged yet):

1. Client download is throttled to the merge upload (slow restores)

The first download of an unmerged entry feeds parts into both the client response and the /merged re-upload through one backpressured pipe (pumpPartsToStreams — await once(mergerStream, 'drain')). The merge upload runs at queueSize: 1, partSize: 5MB (serial 5MB PUTs), so the client is capped to that rate.

Observed on a ~705 MB cache (S3 driver): the restore streams the first ~124 MiB fast (~29 MB/s), then stalls and crawls in ~128 MiB steps with ~30 s pauses — ~3m40s total at ~3–4 MB/s:

Received 130023424 of 739620889 (17.6%), 29.0 MBs/sec   <- then stuck
Received 130023424 of 739620889 (17.6%), 6.5 MBs/sec
Received 264241152 of 739620889 (35.7%), 6.8 MBs/sec    <- +128MiB after ~30s
...
Received 739620889 of 739620889 (100.0%), 3.6 MBs/sec

2. Inline parts deletion races in-flight readers (500s / corruption)

On merge completion the code deletes {folder}/parts inline. Concurrent downloads still streaming those parts then hit ObjectNotFound → 500 (ERR_HTTP_HEADERS_SENT after the response started), which clients treat as a miss → re-upload churn, and entries that never stay merged keep falling back to the slow path forever.

Fix

• Serve the client directly from streamParts() at full speed; run the merge as an independent background stream (its own part reads). The client is never coupled to the merge upload.
• Atomically claim the merge (UPDATE storage_locations SET mergeStartedAt=now WHERE id=? AND mergeStartedAt IS NULL AND mergedAt IS NULL) so concurrent / multi-replica downloads don't start duplicate merges.
• Stop deleting parts inline — the existing hourly cleanup:parts task already removes parts of merged entries, so in-flight readers are never cut off.
• Removes pumpPartsToStreams and downloadFromCacheEntryLocation (no longer needed).

Net: first read of an unmerged entry streams at full speed; once merged, reads use the presigned direct-download path as before. Restore of the same ~705 MB cache dropped from ~3m40s to ~20s in our environment.

Tests

Adds tests/concurrent-download.test.ts: builds an unmerged multi-part entry, fires concurrent downloads, asserts every reader gets the full payload and that parts are not deleted inline (the merge keeps them for cleanup:parts). Fails on the current inline-delete code, passes after. Existing stale-cache + e2e suites pass (sqlite+filesystem and via the server e2e).

Relation to #235

#235 addresses defect 2 (parts-delete race) via a 15-min grace period + per-part existence check — complementary, and its grace approach could be combined. This PR additionally fixes defect 1 (the download-throttled-to-merge stall), which #235 does not touch. Happy to align with #235's approach if preferred.

[peter-svensson]

Re-opening from a personal fork; closing this one.