Repository navigation
Read FITS 16-bit integers as signed and expose BZERO/BSCALE in im.info - #10312
karandhaodiyal28-hash wants to merge 4 commits into
Conversation
FITS integers are signed big-endian, but BITPIX=16 data was unpacked with the unsigned I;16B rawmode, so negative values wrapped around and unsigned camera data (stored as signed with BZERO=32768) came out with its top bit flipped. Read 16-bit data as signed I;16BS into mode I, matching the TIFF plugin, and extend the gzip decoder the same way. The BZERO/BSCALE scaling keywords are now exposed in im.info instead of being silently dropped; applying the scaling is left to the caller. Fixes python-pillow#10309
for more information, see https://pre-commit.ci
|
I independently tested Minimal reproduction using Astropy to generate the compressed file (tested with Astropy 8.0.2 and NumPy 2.5.4): from io import BytesIO
import numpy as np
from astropy.io import fits
from PIL import Image
values = np.array([[-32768, -1, 0, 1, 32767]], dtype=np.int16)
output = BytesIO()
fits.HDUList([
fits.PrimaryHDU(),
fits.CompImageHDU(values, compression_type="GZIP_1", tile_shape=(1, 5)),
]).writeto(output)
blob = output.getvalue()
with fits.open(BytesIO(blob)) as hdus:
hdus.verify("exception")
print("Astropy:", hdus[1].data.tolist())
with Image.open(BytesIO(blob)) as image:
print("Pillow:", [image.getpixel((x, 0)) for x in range(image.width)])Output: The decompressed payload contains consecutive 2-byte samples, but the new branch still takes For comparison, current main Validation on Linux x86_64 / Python 3.14.7: AI disclosure: this investigation, executed reproductions and comment were prepared with OpenAI Codex. The account owner has not manually reviewed the code. |
The decompressed GZIP_1 payload holds consecutive samples, while the existing fixture uses groups widened to 4 bytes. Distinguish the two by the decompressed buffer length, like astropy does, and read 16-bit samples as signed in both layouts. This also replaces the reversed slices, which read past the start of the buffer for 32-bit data. Fixes the case reported in python-pillow#10312
|
Good catch, thanks for the thorough report and the clean reproduction. You're right about the cause — the decoder assumed the widened 4-byte layout and skipped every second sample. I've pushed a fix that distinguishes the payload widths by the decompressed buffer length, like astropy does, and reads 16-bit samples as signed in both layouts. The signed 16-bit compressed regression test is added too (Tests/images/gzip_1_16bit.fits), and m13_gzip still decodes identically. While testing the other bit depths I also found the old reversed slice read past the start of the buffer for 32-bit data, so that's fixed along the way (8/16/32-bit and float32 GZIP_1 files now match astropy on my end). Could you re-check with your fixture? |
FITS integers are signed big-endian, but BITPIX=16 data was unpacked with the unsigned
I;16Brawmode. Negative values wrapped around, and unsigned camera data (stored as signed withBZERO=32768per Section 5.2) came out with its top bit flipped:This implements option 2 from the issue:
I;16BSinto modeI, like the TIFF plugin does for signed 16-bit. The gzip-compressed path (FitsGzipDecoder) is extended the same way — it expands 2-byte samples to the 4-byteIrawmode so both paths agree.BZERO/BSCALEare now exposed inim.info(int when integral, float otherwise, FITSDexponents handled) instead of being silently dropped. Applying the scaling is left to the caller, per FITS Standard 4.0 Section 4.4.2.5.Adds regression tests building the files in memory, following the reproducer from the issue.
Fixes #10309