highlights: fix out-of-bounds stack write on 4-colour CFA sensors

[da-phil]

Within src/iop/hlreconstruct/opposed.c fcol() returns 0-3, but the clipped-photosite counter in the opposed reconstruction was a char[3]. On CYGM/RGBE images the write to mbuff[3] lands one byte past the array and trips the stack protector, aborting with "stack smashing detected" - reproducible with integration test 0177-bayer4 (Canon PowerShot G1).

Only indices 0-2 are ever read back, so widening the array fixes the overflow without changing any output.

The nightly integration test run is green because it's UB: whether that byte hits the canary or padding depends on the reference build's compiler and hardening flags.

This issue showed up when I was running the integration test suite for #22042, see linked test log file with this output:

 Test 0177-bayer4
      Image bayer4.crw
      Timing cpu *** stack smashing detected ***: terminated
*** stack smashing detected ***: terminated
*** stack smashing detected ***: terminated
*** stack smashing detected ***: terminated
8,14s
      Timing gpu *** stack smashing detected ***: terminated
*** stack smashing detected ***: terminated
*** stack smashing detected ***: terminated
*** stack smashing detected ***: terminated
2,82s
  FAILS : darktable-cli errored

Disclaimer: this change was co-created with Claude.

[Copilot]

Pull request overview

Fixes a stack out-of-bounds write in the highlights “opposed” reconstruction path when processing 4-colour CFA sensors (where fcol() may return 3). This prevents intermittent “stack smashing detected” aborts seen in integration testing, without changing the algorithm’s downstream behavior for the RGB channels.

Changes:

• Widen the per-tile clipped-photosite counter buffer from char[3] to char[4] to safely accommodate fcol() values 0..3.
• Add an inline comment documenting why the 4th slot exists (written for CYGM/RGBE patterns, not used in the RGB-channel mask generation).

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[jenshannoschwalm]

Hmm, in short i think this pr is just wrong :-)
Why? The opposed algo just doesn't work with any non-pure-RGB sensor so we don't want a bad workaround but must ensure a fallback to clipping mode.

Will do a PR fixing this ...

[da-phil]

Hmm, in short i think this pr is just wrong :-) Why? The opposed algo just doesn't work with any non-pure-RGB sensor so we don't want a bad workaround but must ensure a fallback to clipping mode.

Will do a PR fixing this ...

Even better, thanks 🙏

I'm still not sure why this issue bubbled up with a Canon PowerShot G1 raw image within the 0177-bayer4 integration test. Isn't this sensor a pure RGB Bayer sensor?

[da-phil]

Okay, turns out I just got confused with the bayer4 naming convention.
A sensor which has more than R, G and B color filters isn't strictly a bayer CFA anymore, right?

[da-phil]

Closing in favor of #22072

[jenshannoschwalm]

Sorry about not being clear, yes a true-bayer sensor has 2 green photosites. The opposed algo doesnt know about other colors than RGB