Skip to content

Prevent negative ImageStat variance from floating-point rounding - #9994

Open
lllleolin-max wants to merge 2 commits into
python-pillow:mainfrom
lllleolin-max:fix/stat-variance
Open

Prevent negative ImageStat variance from floating-point rounding#9994
lllleolin-max wants to merge 2 commits into
python-pillow:mainfrom
lllleolin-max:fix/stat-variance

Conversation

@lllleolin-max

@lllleolin-max lllleolin-max commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

On affected platforms, for a constant white image of size (919, 405), floating-point rounding makes ImageStat.Stat.var slightly negative (-1.0249189982737543e-11). Accessing stddev then raises ValueError: math domain error.

Changes proposed in this pull request:

  • Clamp the calculated variance to zero, so roundoff cannot make it negative or cause the subsequent square root to fail.
  • Add regressions for grayscale and RGB images, masked regions and precalculated histograms, plus a nonzero-variance control. Reject negative variance while allowing small positive roundoff, which occurs on macOS, PyPy and 32-bit CI. The tolerance is four ULPs at the squared 8-bit pixel scale, with the corresponding square-root tolerance for standard deviation. Include a release note.

Reproduction:

from PIL import Image, ImageStat

stats = ImageStat.Stat(Image.new("L", (919, 405), 255))
print(stats.var)
print(stats.stddev)

Validation:

  • New regression tests against the unchanged main implementation: 4 failed, 5 passed. After the fix, Tests/test_imagestat.py, Tests/test_imageops.py and Tests/test_image_histogram.py: 59 passed, no skips.
  • selftest.py: 59 tests passed.
  • Black, Ruff, mypy, Bandit, Sphinx-lint and whitespace/end-of-file/line-ending checks passed for the changed files.
  • A local floating-point perturbation through the production calculation reproduces CI's positive variance (1.0249189982737543e-11); all four image/mask/histogram cases pass with it. The unchanged implementation still fails all four cases for negative roundoff.
  • Parsed the release-note heading with Docutils: the published short underline reproduces the Sphinx warning; the corrected underline parses without warnings.

Tests ran on Windows/Python 3.12.11 using the checkout's ImageStat and ImageOps Python modules with Pillow 12.3.0's prebuilt native extensions and remaining Python modules. Native extensions were not rebuilt; the full test suite, full Sphinx build and full pre-commit suite were not run locally. The positive-roundoff replay is a local simulation, not a claim of local macOS or PyPy execution.

Prepared with AI assistance.

@radarhere radarhere added the 🤖-assisted AI-assisted label Sep 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🤖-assisted AI-assisted

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants