fix(test): stop comparing rows that no walk could measure - #43
Merged
Conversation
ChunkFootprintTest failed three times on 2026-08-03 -- twice in pull requests and once on main, where it stopped the snapshot from being published. Two different test methods, both at the same line, both with a count below zero: FalcoChunk retains -2 objects of [B against 0 of DynamicChunk FalcoChunk retains 0 objects of java.io.FileCleanable against -3 of DynamicChunk The per class figures are differences between two walks: everything the chunk and its instance retain, minus what the instance retains alone. A count below zero cannot be a property of the chunk -- a set of objects that exist because the chunk exists has no negative cardinality. It means the instance grew between the two walks, and both classes seen doing it are the JVM's own bookkeeping: FileCleanable when a file handle is registered, byte[] when a string is built. Comparing such a row states nothing about either chunk. Those rows are now skipped, their bytes carried into the total so the table still has to sum to what the footprint reports, and the classes printed. The exclusion is capped at two classes, and the cap is the part that matters. Without it the mechanism passes everything: inverting the condition so that every class counts as unmeasurable left the test green, because the byte correction absorbed the whole comparison. With the cap the same edit fails. Skipping one or two rows loses nothing; skipping many means the walk disagreed with itself, and the answer to that is a failure rather than a comparison of what survived. Both failures carried exactly one such class. The expectations are untouched. 25 objects and 840 bytes are still asserted exactly, and no tolerance was introduced anywhere. Checked by injecting a real defect -- an Object[4] field on FalcoChunk -- which fails as it did before, and reverted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NGpJqdmh7ZNH487GLqPJmK
Contributor
Test results 267 files 267 suites 10m 31s ⏱️ Results for commit ff0cf63. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
ChunkFootprintTestfailed three times on 2026-08-03 — in #39, in #40, and onmain, where it stopped the snapshot from being published. Two different test methods, both at the same line, both with a count below zero:The per-class figures are differences between two walks: everything the chunk and its instance retain, minus what the instance retains alone. A count below zero cannot be a property of the chunk — a set of objects that exist because the chunk exists has no negative cardinality. It means the instance grew between the two walks, and both classes caught doing it are the JVM's own bookkeeping:
FileCleanablewhen a file handle is registered,byte[]when a string is built. Comparing such a row states nothing about either chunk.Roughly three of ten CI runs, on ubuntu and on windows. Never reproduced locally in 8 attempts, including 3 pinned to two cores.
What changed
Rows with a negative count on either side are skipped, their bytes carried into the total so the per-class table still has to sum to what the footprint reports, and the classes are printed.
The cap of two is the part that matters. Without it the mechanism passes everything: inverting the condition so that every class counts as unmeasurable left the test green, because the byte correction absorbed the whole comparison. That is the failure mode this repository has paid for repeatedly — a check that cannot fail. With the cap, the same edit fails. Both observed failures carried exactly one such class.
What did not change
The expectations. 25 objects and 840 bytes are still asserted exactly, and no tolerance was introduced anywhere.
Checks that it still bites
Object[4]field onFalcoChunkWhat this does not do
It does not stop the JVM from allocating between the two walks — it stops that from being reported as a property of the chunk. The underlying non-determinism of a two-walk measurement is unchanged, and if a future failure names a class that is not JVM bookkeeping, this cap will surface it rather than hide it.