repository: do not crash on missing segment files/dirs, fixes #3225

[ThomasWaldmann]

Two crashes on repositories that lost segment files or segment dirs (e.g. removed by a filesystem check), both reported in #3225.

Writing into a missing segment dir

LoggedIO.get_write_fd only created the segment dir if the segment was the first one of that dir (segment % segments_per_dir == 0), so writing a segment file into a dir that is not there any more crashed with FileNotFoundError. borg check --repair runs into this when it adds a commit tag to segment <index transaction id> + 1 and that segment's dir is gone - so repair died on the damage it is meant to repair. The dir is now created whenever it is missing.

Reading a missing segment file

LoggedIO.get_fd let a FileNotFoundError escape when the repository index still referred to a segment file that is gone, so borg extract / borg mount / the archives check just dumped a traceback. It now raises an IntegrityError that names the missing segment file and points at borg check --repair:

Data integrity error: Segment file /srv/borg/repo/data/1/3 is missing. Run borg
check (full check, not --archives-only) and then borg check --repair to fix the
repository.

The archives check catches IntegrityError around Repository.get, so borg check --verify-data now logs the affected chunks and continues (and deletes them with --repair, so the files can be repaired and later healed) instead of aborting.

Both tests fail with FileNotFoundError without the fix.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.12%. Comparing base (1b7d327) to head (bf3baf5).
⚠️ Report is 13 commits behind head on 1.4-maint.

Additional details and impacted files

@@              Coverage Diff              @@
##           1.4-maint   #10112      +/-   ##
=============================================
- Coverage      82.62%   82.12%   -0.51%     
=============================================
  Files             38       38              
  Lines          11432    11435       +3     
  Branches        1803     1802       -1     
=============================================
- Hits            9446     9391      -55     
- Misses          1408     1460      +52     
- Partials         578      584       +6     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.