Skip to content

export-tar: support sparse files (GNU sparse format 1.0) #10128

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ThomasWaldmann merged 1 commit into from
Aug 18, 2026
Merged

export-tar: support sparse files (GNU sparse format 1.0) #10128

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 50 additions & 34 deletions docs/usage/export-tar.rst.inc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,39 +12,41 @@ borg export-tar

.. class:: borg-options-table

+-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
| **positional arguments** |
+-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
| | ``NAME`` | specify the archive name |
+-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
| | ``FILE`` | output tar file. "-" to write to stdout instead. |
+-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
| | ``PATH`` | paths to extract; patterns are supported |
+-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
| **options** |
+-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
| | ``--tar-filter`` | filter program to pipe data through |
+-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
| | ``--list`` | output verbose list of items (files, dirs, ...) |
+-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
| | ``--tar-format FMT`` | select tar format: BORG, PAX or GNU |
+-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
| .. class:: borg-common-opt-ref |
| |
| :ref:`common_options` |
+-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
| **Include/Exclude options** |
+-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
| | ``-e PATTERN``, ``--exclude PATTERN`` | exclude paths matching PATTERN |
+-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
| | ``--exclude-from EXCLUDEFILE`` | read exclude patterns from EXCLUDEFILE, one per line |
+-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
| | ``--pattern PATTERN`` | include/exclude paths matching PATTERN |
+-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
| | ``--patterns-from PATTERNFILE`` | read include/exclude patterns from PATTERNFILE, one per line |
+-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
| | ``--strip-components NUMBER`` | Remove the specified number of leading path elements. Paths with fewer elements will be silently skipped. |
+-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
+-------------------------------------------------------+---------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| **positional arguments** |
+-------------------------------------------------------+---------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| | ``NAME`` | specify the archive name |
+-------------------------------------------------------+---------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| | ``FILE`` | output tar file. "-" to write to stdout instead. |
+-------------------------------------------------------+---------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| | ``PATH`` | paths to extract; patterns are supported |
+-------------------------------------------------------+---------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| **options** |
+-------------------------------------------------------+---------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| | ``--tar-filter`` | filter program to pipe data through |
+-------------------------------------------------------+---------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| | ``--list`` | output verbose list of items (files, dirs, ...) |
+-------------------------------------------------------+---------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| | ``--tar-format FMT`` | select tar format: BORG, PAX or GNU |
+-------------------------------------------------------+---------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| | ``--sparse`` | write sparse tar members (GNU sparse format 1.0) for files containing all-zero chunks (BORG and PAX formats only) |
+-------------------------------------------------------+---------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| .. class:: borg-common-opt-ref |
| |
| :ref:`common_options` |
+-------------------------------------------------------+---------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| **Include/Exclude options** |
+-------------------------------------------------------+---------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| | ``-e PATTERN``, ``--exclude PATTERN`` | exclude paths matching PATTERN |
+-------------------------------------------------------+---------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| | ``--exclude-from EXCLUDEFILE`` | read exclude patterns from EXCLUDEFILE, one per line |
+-------------------------------------------------------+---------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| | ``--pattern PATTERN`` | include/exclude paths matching PATTERN |
+-------------------------------------------------------+---------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| | ``--patterns-from PATTERNFILE`` | read include/exclude patterns from PATTERNFILE, one per line |
+-------------------------------------------------------+---------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| | ``--strip-components NUMBER`` | Remove the specified number of leading path elements. Paths with fewer elements will be silently skipped. |
+-------------------------------------------------------+---------------------------------------+-------------------------------------------------------------------------------------------------------------------+

.. raw:: html

Expand All @@ -68,6 +70,7 @@ borg export-tar
--tar-filter filter program to pipe data through
--list output verbose list of items (files, dirs, ...)
--tar-format FMT select tar format: BORG, PAX or GNU
--sparse write sparse tar members (GNU sparse format 1.0) for files containing all-zero chunks (BORG and PAX formats only)


:ref:`common_options`
Expand Down Expand Up @@ -120,7 +123,20 @@ Depending on the ``--tar-format`` option, these formats are created:
| | | no ACLs/xattrs/bsdflags |
+--------------+---------------------------+----------------------------+

A ``--sparse`` option (as found in borg extract) is not supported.
With ``--sparse``, files whose content contains runs of all-zero chunks are written
as sparse tar members (GNU sparse format 1.0, as GNU tar creates it in POSIX mode),
storing only a hole map and the non-zero data. This requires ``--tar-format BORG``
or ``PAX``. Such tarballs can be much smaller for sparse files (e.g. disk images)
and extract to sparse files again with GNU tar's or bsdtar's sparse support
(as well as with ``borg import-tar`` / ``borg extract --sparse``).
Notes: hole detection works at the granularity of borg's content chunks (it does not
depend on the original file having been a sparse file - but some short or unaligned
zero runs may be stored literally); sparse-unaware tar implementations will extract
a member as ``GNUSparseFile.0/<name>`` containing the raw hole map and data (the
same caveat applies to tarballs created by GNU tar); for members needing >= 8 GiB
of stored (non-hole) data, the stored size is base-256 encoded in the tar header
(the GNU/star encoding of big numbers, understood by GNU tar, libarchive/bsdtar
and python) - logical file sizes are unlimited anyway.

By default the entire archive is extracted but a subset of files and directories
can be selected by passing a list of ``PATHs`` as arguments.
Expand Down
4 changes: 3 additions & 1 deletion docs/usage/import-tar.rst.inc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,9 @@ stdout.
Most documentation of borg create applies. Note that this command does not
support excluding files.

A ``--sparse`` option (as found in borg create) is not supported.
A ``--sparse`` option (as found in borg create) is not needed: sparse members in
input tarballs (old GNU and PAX sparse formats) are read correctly and their
holes are stored as deduplicated all-zero chunks.

About tar formats and metadata conservation or loss, please see ``borg export-tar``.

Expand Down
6 changes: 5 additions & 1 deletion docs/usage/tar.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ Examples
# Remote extraction via 'tarpipe'
$ borg export-tar Monday - | ssh somewhere "cd extracted; tar x"

# Export sparse files (e.g. disk images) as sparse tar members (GNU sparse format 1.0)
$ borg export-tar --sparse disk-images disk-images.tar

Archives transfer script
~~~~~~~~~~~~~~~~~~~~~~~~

Expand Down Expand Up @@ -54,4 +57,5 @@ Please note:
- the pipe is dumb, there is no data or transfer time reduction there due to deduplication
- maybe add compression
- pipe over ssh for remote transfer
- no special sparse file support
- maybe add ``--sparse`` to the export-tar command, so runs of all-zero chunks
travel as a compact sparse map instead of literal zeros
Loading
Loading