Skip to content

Read FITS 16-bit integers as signed and expose BZERO/BSCALE in im.info - #10312

Open
karandhaodiyal28-hash wants to merge 4 commits into
python-pillow:mainfrom
karandhaodiyal28-hash:fix-fits-16bit-signedness
Open

karandhaodiyal28-hash wants to merge 4 commits into
python-pillow:mainfrom
karandhaodiyal28-hash:fix-fits-16bit-signedness

Conversation

@karandhaodiyal28-hash

Copy link
Copy Markdown

FITS integers are signed big-endian, but BITPIX=16 data was unpacked with the unsigned I;16B rawmode. Negative values wrapped around, and unsigned camera data (stored as signed with BZERO=32768 per Section 5.2) came out with its top bit flipped:

# BITPIX=16, values [-32768, -32767, -1, 0, 32767]
# before: [128, 384, 65535, 0, 65407]
# after:  [-32768, -32767, -1, 0, 32767]

This implements option 2 from the issue:

  • BITPIX=16 is now read as signed I;16BS into mode I, like the TIFF plugin does for signed 16-bit. The gzip-compressed path (FitsGzipDecoder) is extended the same way — it expands 2-byte samples to the 4-byte I rawmode so both paths agree.
  • BZERO/BSCALE are now exposed in im.info (int when integral, float otherwise, FITS D exponents handled) instead of being silently dropped. Applying the scaling is left to the caller, per FITS Standard 4.0 Section 4.4.2.5.

Adds regression tests building the files in memory, following the reproducer from the issue.

Fixes #10309

karandhaodiyal28-hash and others added 3 commits October 25, 2026 22:21
FITS integers are signed big-endian, but BITPIX=16 data was unpacked
with the unsigned I;16B rawmode, so negative values wrapped around and
unsigned camera data (stored as signed with BZERO=32768) came out with
its top bit flipped. Read 16-bit data as signed I;16BS into mode I,
matching the TIFF plugin, and extend the gzip decoder the same way.

The BZERO/BSCALE scaling keywords are now exposed in im.info instead of
being silently dropped; applying the scaling is left to the caller.

Fixes python-pillow#10309
@laozidabo

Copy link
Copy Markdown

I independently tested 2c2333b2e81f9300e93614e3a682892cd4454c15 and found an uncovered case in the changed 16-bit gzip path. A single-tile GZIP_1 image with samples [-32768, -1, 0, 1, 32767] loads as [-1, 1, 0, 0, 0] on this PR.

Minimal reproduction using Astropy to generate the compressed file (tested with Astropy 8.0.2 and NumPy 2.5.4):

from io import BytesIO

import numpy as np
from astropy.io import fits
from PIL import Image

values = np.array([[-32768, -1, 0, 1, 32767]], dtype=np.int16)
output = BytesIO()
fits.HDUList([
    fits.PrimaryHDU(),
    fits.CompImageHDU(values, compression_type="GZIP_1", tile_shape=(1, 5)),
]).writeto(output)
blob = output.getvalue()

with fits.open(BytesIO(blob)) as hdus:
    hdus.verify("exception")
    print("Astropy:", hdus[1].data.tolist())
with Image.open(BytesIO(blob)) as image:
    print("Pillow:", [image.getpixel((x, 0)) for x in range(image.width)])

Output:

Astropy: [[-32768, -1, 0, 1, 32767]]
Pillow: [-1, 1, 0, 0, 0]

The decompressed payload contains consecutive 2-byte samples, but the new branch still takes value[offset + 2 : offset + 4] and increments offset by 4. It skips samples and eventually calls int.from_bytes() on empty slices, producing the trailing zeros. Could this two-byte payload be handled with a signed 16-bit compressed regression test, while retaining support for the widened four-byte payload used by existing fixtures? Astropy's decoder also distinguishes these widths by decompressed buffer length. I also reproduced this with a standard-library-built FITS fixture, independently of Astropy's writer.

For comparison, current main f58197e0effb8e650b9f50f858ec8976e6f6a83d raises ValueError: not enough image data on the standard-library fixture. So gzip 16-bit support was already incomplete; the PR changes this case to a successful load with incorrect pixels, rather than breaking a previously correct decode.

Validation on Linux x86_64 / Python 3.14.7: selftest.py passes all 59 tests on both revisions; Tests/test_file_fits.py passes 8 tests on main and 11 on this PR. The full suite was not run for this focused investigation. No source patch is being claimed here.

AI disclosure: this investigation, executed reproductions and comment were prepared with OpenAI Codex. The account owner has not manually reviewed the code.

The decompressed GZIP_1 payload holds consecutive samples, while the
existing fixture uses groups widened to 4 bytes. Distinguish the two by
the decompressed buffer length, like astropy does, and read 16-bit
samples as signed in both layouts. This also replaces the reversed
slices, which read past the start of the buffer for 32-bit data.

Fixes the case reported in python-pillow#10312
@karandhaodiyal28-hash

Copy link
Copy Markdown
Author

Good catch, thanks for the thorough report and the clean reproduction. You're right about the cause — the decoder assumed the widened 4-byte layout and skipped every second sample.

I've pushed a fix that distinguishes the payload widths by the decompressed buffer length, like astropy does, and reads 16-bit samples as signed in both layouts. The signed 16-bit compressed regression test is added too (Tests/images/gzip_1_16bit.fits), and m13_gzip still decodes identically.

While testing the other bit depths I also found the old reversed slice read past the start of the buffer for 32-bit data, so that's fixed along the way (8/16/32-bit and float32 GZIP_1 files now match astropy on my end). Could you re-check with your fixture?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FITS: BZERO/BSCALE are ignored and 16-bit data is read as unsigned

2 participants