From fc793b0b4f76076298dd3fb5b030c746504a8e36 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 5 Nov 2026 06:30:36 +0000 Subject: [PATCH 1/9] Charge alignment padding against the destination hole when moving objects With HXCPP_GC_MOVING and HXCPP_ALIGN_ALLOC (the default), MoveBlocks and MoveSurvivors reserve room for the alignment padding of each moved object in their fit test, add the padding to destPos, but only subtract allocSize from destLen. The hole length therefore over-counts the free space by the padding of every object placed so far, and once a destination hole has been filled with enough moved objects, the next one is copied past the end of the hole - over the next block in the group, including its block id and row marks. The following mark then reads corrupt headers and crashes. Charge the padding against destLen as well, so that destPos + destLen stays equal to the end of the hole. The same loop in MoveSurvivors (currently unused) gets the same fix, and loses a stray printf. A regression test allocates a linked set of small objects and compacts the heap several times; with HXCPP_GC_MOVING it crashed before this change. The reproducer from #1167 segfaulted on every run before and completes after. Fixes #1167, fixes #1414. The same fix was made downstream in openfl/libhxcpp@24cd6118. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Va4FXGUp4C1CTLzsj6xMjY --- Changes.md | 1 + src/hx/gc/Immix.cpp | 18 +++++++++++++++--- test/haxe/gc/TestGC.hx | 28 ++++++++++++++++++++++++++++ 3 files changed, 44 insertions(+), 3 deletions(-) diff --git a/Changes.md b/Changes.md index 816b4be7c..1de32727b 100644 --- a/Changes.md +++ b/Changes.md @@ -21,6 +21,7 @@ * Fixed critial error handler returning the wrong callstack * Fixed ARM64 library names on Mac * Fixed generational GC when used with HXCPP_ALIGN_ALLOC +* Fixed heap corruption in the moving GC (HXCPP_GC_MOVING) with HXCPP_ALIGN_ALLOC: compaction did not charge the alignment padding of each moved object against the destination hole, so a block filled by moved objects overran its end * Fixed pthread structured being unaligned * Fixed cppia crash on functions with empty bodies * Fixed regression parsing integers which wrap around diff --git a/src/hx/gc/Immix.cpp b/src/hx/gc/Immix.cpp index 367b2760a..e49136e1c 100644 --- a/src/hx/gc/Immix.cpp +++ b/src/hx/gc/Immix.cpp @@ -3802,7 +3802,14 @@ class GlobalAllocator } #ifdef HXCPP_ALIGN_ALLOC - destPos += ALIGN_PADDING(destPos); + // The fit test above reserved allocSize + padding, so the padding has to + // come out of destLen as well, or destPos + destLen drifts past the end + // of the hole and a later object is written over the next block + { + int alignPad = ALIGN_PADDING(destPos); + destPos += alignPad; + destLen -= alignPad; + } #endif int startRow = destPos>>IMMIX_LINE_BITS; @@ -4010,9 +4017,14 @@ class GlobalAllocator // TODO - not copy + paste - printf("Move!\n"); #ifdef HXCPP_ALIGN_ALLOC - destPos += ALIGN_PADDING(destPos); + // As in MoveBlocks: the padding was reserved by the fit test, so charge + // it against destLen too + { + int alignPad = ALIGN_PADDING(destPos); + destPos += alignPad; + destLen -= alignPad; + } #endif int startRow = destPos>>IMMIX_LINE_BITS; diff --git a/test/haxe/gc/TestGC.hx b/test/haxe/gc/TestGC.hx index 5b297d27a..904d0aa70 100644 --- a/test/haxe/gc/TestGC.hx +++ b/test/haxe/gc/TestGC.hx @@ -9,6 +9,15 @@ class CustomObject { public function new():Void {} } +class CompactValue { + public var id:Int; + public var prev:CompactValue; + public function new(id:Int, prev:CompactValue) { + this.id = id; + this.prev = prev; + } +} + class TestGC extends Test { function createDummy(val:Dynamic):Dynamic { return { dummy: val }; @@ -138,6 +147,25 @@ class TestGC extends Test { } #end + // A compaction moves live objects into other blocks. With HXCPP_GC_MOVING and the default + // HXCPP_ALIGN_ALLOC, the alignment padding of each moved object was not charged against the + // destination hole, so a block filled by moved objects overran its end and corrupted the next + // block (#1167, #1414). Without HXCPP_GC_MOVING this is an ordinary full collection. + public function testCompactKeepsObjects() { + var prev:CompactValue = null; + final values = [for (i in 0...200000) prev = new CompactValue(i, prev)]; + for (pass in 0...3) { + Gc.compact(); + var bad = 0; + for (i in 0...values.length) { + final value = values[i]; + if (value.id != i || value.prev != (i == 0 ? null : values[i - 1])) + bad++; + } + Assert.equals(0, bad, 'Compaction pass $pass corrupted $bad objects'); + } + } + public function testLargeAlloc() { var largeString = { var bytes = Bytes.alloc(4000); From b9994165a25776e6cc8b24e53daa76a6bee5d873 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 5 Nov 2026 06:30:50 +0000 Subject: [PATCH 2/9] Initialise the thread's mark id copy when a StackContext is created Under HXCPP_GC_GENERATIONAL the cppia jit's write barrier compares an object's mark byte with StackContext::byteMarkId, the thread's copy of the collector's mark id. The collector refreshes that copy at the end of every collection and RegisterCurrentThread sets it for new threads, but the main thread's context is created in InitAlloc with the field uninitialised, and nothing sets it until the first collection has run. While it reads 0, every nursery object - whose mark byte is 0 until it survives a collection - looks like an old object to the barrier, which flags it remembered and pushes it as a referrer. A flagged nursery object no longer reads as a nursery header, so the conservative stack scan cannot trace past it to the nursery objects allocated after it in the same hole, and a collection frees those while the stack still holds them. A script method called through Reflect.callMethod that ran Gc.compact() lost the closure it was running in (#1196). Set byteMarkId from hx::gByteMarkID in the StackContext constructor so every context starts in step with the collector. Fixes #1196. Same change as openfl/libhxcpp@0fe6cbee. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Va4FXGUp4C1CTLzsj6xMjY --- Changes.md | 1 + src/hx/Debug.cpp | 7 +++++++ 2 files changed, 8 insertions(+) diff --git a/Changes.md b/Changes.md index 1de32727b..fe7d6e7f8 100644 --- a/Changes.md +++ b/Changes.md @@ -22,6 +22,7 @@ * Fixed ARM64 library names on Mac * Fixed generational GC when used with HXCPP_ALIGN_ALLOC * Fixed heap corruption in the moving GC (HXCPP_GC_MOVING) with HXCPP_ALIGN_ALLOC: compaction did not charge the alignment padding of each moved object against the destination hole, so a block filled by moved objects overran its end +* Fixed the cppia jit write barrier treating new objects as old on the main thread until the first collection, which could let a collection free objects that only the stack held (#1196) * Fixed pthread structured being unaligned * Fixed cppia crash on functions with empty bodies * Fixed regression parsing integers which wrap around diff --git a/src/hx/Debug.cpp b/src/hx/Debug.cpp index c49cc935a..c5f437d7a 100644 --- a/src/hx/Debug.cpp +++ b/src/hx/Debug.cpp @@ -207,6 +207,13 @@ StackContext::StackContext() #endif #ifdef HXCPP_SCRIPTABLE + // The cppia jit's write barrier compares an object's mark byte with this copy of the + // collector's mark id, so it must match from the start. The collector only updates it at + // the end of each collection, and the main thread's context is created before any of them. + // Left uninitialised (0 in practice), every nursery object - whose mark byte is 0 - looked + // old to the barrier, which flagged it remembered and so hid the nursery objects allocated + // after it from the stack scan (#1196). + byteMarkId = ::hx::gByteMarkID; stack = new unsigned char[128*1024]; pointer = &stack[0]; push((hx::Object *)0); From 03e37b0d36ec7ea283d2bc16ad61fff71ffbd5df Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 5 Nov 2026 06:32:45 +0000 Subject: [PATCH 3/9] Keep the GC verify table update ahead of the mark id wrap The 4-bit mark id cycles through 16 values. Dead objects keep their last mark id in the header until a full reclaim clears their allocStart bits, so the "table update" must happen before the id comes back round: without HXCPP_GC_MOVING, sgTimeToNextTableUpdate is reset to 15 and decremented once per non-generational mark, which forces a full reclaim on the 15th mark after the last one, one short of the wrap. With HXCPP_GC_VERIFY and HXCPP_GC_GENERATIONAL (HX_GC_VERIFY_GENERATIONAL) every collection runs a second, non-generational verification mark, which advances the id and decrements the counter a second time. Starting from 15, the eighth collection then performs the 15th and 16th marks before the counter is checked: the id wraps, the verification pass finds objects that died up to 16 marks ago with a header equal to the current id, and reports "Unmarked row ... After mark". Reset the counter to 14 in that build so the full reclaim happens on or before the 15th mark again. Builds without HXCPP_GC_VERIFY are unchanged. Refs #1415 (the TestGCThreaded failure with HXCPP_GC_GENERATIONAL). Same change as openfl/libhxcpp@f902b9d9. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Va4FXGUp4C1CTLzsj6xMjY --- Changes.md | 1 + src/hx/gc/Immix.cpp | 4 ++++ 2 files changed, 5 insertions(+) diff --git a/Changes.md b/Changes.md index fe7d6e7f8..b9b6184b2 100644 --- a/Changes.md +++ b/Changes.md @@ -23,6 +23,7 @@ * Fixed generational GC when used with HXCPP_ALIGN_ALLOC * Fixed heap corruption in the moving GC (HXCPP_GC_MOVING) with HXCPP_ALIGN_ALLOC: compaction did not charge the alignment padding of each moved object against the destination hole, so a block filled by moved objects overran its end * Fixed the cppia jit write barrier treating new objects as old on the main thread until the first collection, which could let a collection free objects that only the stack held (#1196) +* Fixed HXCPP_GC_VERIFY reporting an unmarked row with the generational collector: the verification mark advanced the mark id twice per collection, so dead objects from 16 marks earlier could read as marked before the next table update cleared them (#1415) * Fixed pthread structured being unaligned * Fixed cppia crash on functions with empty bodies * Fixed regression parsing integers which wrap around diff --git a/src/hx/gc/Immix.cpp b/src/hx/gc/Immix.cpp index e49136e1c..191459586 100644 --- a/src/hx/gc/Immix.cpp +++ b/src/hx/gc/Immix.cpp @@ -5058,6 +5058,10 @@ class GlobalAllocator { #ifdef HXCPP_GC_MOVING sgTimeToNextTableUpdate = 7; + #elif defined(HX_GC_VERIFY_GENERATIONAL) + // The verification mark advances the mark id a second time in each collection, so the + // count has to come down one sooner to keep the table update ahead of the 16 mark ids + sgTimeToNextTableUpdate = 14; #else sgTimeToNextTableUpdate = 15; #endif From ce5b3f7d53fa6ba2084a1d797bc01ef37b9594a4 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 5 Nov 2026 06:32:45 +0000 Subject: [PATCH 4/9] Erase the old object id entry before inserting the moved address MoveSpecial transfers an object's id to its new address by inserting sObjectIdMap[inTo] and then erasing the iterator that find() returned for the old address. An insert into std::unordered_map may rehash, and a rehash invalidates all iterators, so the erase could operate on a dangling iterator. The other maps in MoveSpecial already erase before they insert; do the same here. Only HXCPP_GC_MOVING builds reach this code; 64-bit builds use the object id map for every ObjectMap key, so a compaction of a program with many object keys could hit a rehash at exactly the wrong moment. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Va4FXGUp4C1CTLzsj6xMjY --- Changes.md | 1 + src/hx/gc/Immix.cpp | 6 ++++-- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/Changes.md b/Changes.md index b9b6184b2..046fce184 100644 --- a/Changes.md +++ b/Changes.md @@ -24,6 +24,7 @@ * Fixed heap corruption in the moving GC (HXCPP_GC_MOVING) with HXCPP_ALIGN_ALLOC: compaction did not charge the alignment padding of each moved object against the destination hole, so a block filled by moved objects overran its end * Fixed the cppia jit write barrier treating new objects as old on the main thread until the first collection, which could let a collection free objects that only the stack held (#1196) * Fixed HXCPP_GC_VERIFY reporting an unmarked row with the generational collector: the verification mark advanced the mark id twice per collection, so dead objects from 16 marks earlier could read as marked before the next table update cleared them (#1415) +* Fixed the moving GC updating the object id map through an iterator that the insert of the new address could have invalidated * Fixed pthread structured being unaligned * Fixed cppia crash on functions with empty bodies * Fixed regression parsing integers which wrap around diff --git a/src/hx/gc/Immix.cpp b/src/hx/gc/Immix.cpp index 191459586..b2941d851 100644 --- a/src/hx/gc/Immix.cpp +++ b/src/hx/gc/Immix.cpp @@ -3704,9 +3704,11 @@ class GlobalAllocator hx::ObjectIdMap::iterator id = hx::sObjectIdMap.find(inFrom); if (id!=hx::sObjectIdMap.end()) { - hx::sIdObjectMap[id->second] = inTo; - hx::sObjectIdMap[inTo] = id->second; + int objectId = id->second; + // Erase before inserting - the insert can rehash the map and invalidate the iterator hx::sObjectIdMap.erase(id); + hx::sObjectIdMap[inTo] = objectId; + hx::sIdObjectMap[objectId] = inTo; } // Maybe do something faster with weakmaps From eaae37bc570b7accdeda4a110bf51abcb8bd9e5a Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 5 Nov 2026 06:32:45 +0000 Subject: [PATCH 5/9] Reject an object id equal to the id table size GetIDObject guarded against inIndex > size() but let inIndex == size() through, which reads the element just past the end of sIdObjectMap (QuickVec does no bounds checking) and returns whatever happens to be there as an object pointer. Use >= like every other bounds check. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Va4FXGUp4C1CTLzsj6xMjY --- Changes.md | 1 + src/hx/gc/Immix.cpp | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/Changes.md b/Changes.md index 046fce184..81ea093dd 100644 --- a/Changes.md +++ b/Changes.md @@ -25,6 +25,7 @@ * Fixed the cppia jit write barrier treating new objects as old on the main thread until the first collection, which could let a collection free objects that only the stack held (#1196) * Fixed HXCPP_GC_VERIFY reporting an unmarked row with the generational collector: the verification mark advanced the mark id twice per collection, so dead objects from 16 marks earlier could read as marked before the next table update cleared them (#1415) * Fixed the moving GC updating the object id map through an iterator that the insert of the new address could have invalidated +* Fixed __hxcpp_id_obj reading one past the end of the id table for an id equal to its size * Fixed pthread structured being unaligned * Fixed cppia crash on functions with empty bodies * Fixed regression parsing integers which wrap around diff --git a/src/hx/gc/Immix.cpp b/src/hx/gc/Immix.cpp index b2941d851..ac2b4fd74 100644 --- a/src/hx/gc/Immix.cpp +++ b/src/hx/gc/Immix.cpp @@ -4218,7 +4218,7 @@ class GlobalAllocator void *GetIDObject(int inIndex) { std::lock_guard lock(*gSpecialObjectLock); - if (inIndex<0 || inIndex>hx::sIdObjectMap.size()) + if (inIndex<0 || inIndex>=hx::sIdObjectMap.size()) return 0; return hx::sIdObjectMap[inIndex]; } From 7cdb2b0c04783cac35c43c278920ab5ac27808a3 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 5 Nov 2026 06:32:45 +0000 Subject: [PATCH 6/9] Compute the large allocation target from the working memory as size_t Collect derived the large-object collection target from mAllBlocks.size() << IMMIX_BLOCK_BITS in an int. With HXCPP_GC_BIG_BLOCKS (64 KB blocks, 32-bit ids) a heap of more than 32768 blocks - 2 GB - makes that shift negative; converted to size_t for the comparison it becomes huge, so mLargeAllocSpace and mLargeAllocForceRefresh are set to garbage and large allocations either collect on every call or never trigger a collection again. Use GetWorkingMemory(), which already does this computation in size_t, and do the same in the two verbose logging lines that had the same shift. This is the fix from PR #1339 by Tutez64. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Va4FXGUp4C1CTLzsj6xMjY --- Changes.md | 1 + src/hx/gc/Immix.cpp | 6 +++--- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/Changes.md b/Changes.md index 81ea093dd..8755da7fa 100644 --- a/Changes.md +++ b/Changes.md @@ -26,6 +26,7 @@ * Fixed HXCPP_GC_VERIFY reporting an unmarked row with the generational collector: the verification mark advanced the mark id twice per collection, so dead objects from 16 marks earlier could read as marked before the next table update cleared them (#1415) * Fixed the moving GC updating the object id map through an iterator that the insert of the new address could have invalidated * Fixed __hxcpp_id_obj reading one past the end of the id table for an id equal to its size +* Fixed the large allocation collection threshold overflowing once the block heap passes 2GB with HXCPP_GC_BIG_BLOCKS * Fixed pthread structured being unaligned * Fixed cppia crash on functions with empty bodies * Fixed regression parsing integers which wrap around diff --git a/src/hx/gc/Immix.cpp b/src/hx/gc/Immix.cpp index ac2b4fd74..407fa7dda 100644 --- a/src/hx/gc/Immix.cpp +++ b/src/hx/gc/Immix.cpp @@ -3551,11 +3551,11 @@ class GlobalAllocator #if defined(SHOW_MEM_EVENTS_VERBOSE) || defined(SHOW_FRAGMENTATION_BLOCKS) if (inJustBorrowing) { - GCLOG("Borrow Blocks %d = %d k\n", mAllBlocks.size(), (mAllBlocks.size() << IMMIX_BLOCK_BITS)>>10); + GCLOG("Borrow Blocks %d = %d k\n", mAllBlocks.size(), (int)(GetWorkingMemory()>>10)); } else { - GCLOG("Blocks %d = %d k\n", mAllBlocks.size(), (mAllBlocks.size() << IMMIX_BLOCK_BITS)>>10); + GCLOG("Blocks %d = %d k\n", mAllBlocks.size(), (int)(GetWorkingMemory()>>10)); } #endif @@ -5269,7 +5269,7 @@ class GlobalAllocator #endif // Large alloc target - int blockSize = mAllBlocks.size()< mLargeAllocSpace) mLargeAllocSpace = blockSize; mLargeAllocForceRefresh = mLargeAllocated + mLargeAllocSpace; From c81f4b2e0e7cb3f6b68333fbeaf9c474cdc2e193 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 5 Nov 2026 06:33:38 +0000 Subject: [PATCH 7/9] Decide a new thread's referrer chunk under the thread state lock With HXCPP_GC_GENERATIONAL every registered allocator owns a MarkChunk (mOldReferrers) while the collector is in generational mode, and none while it is in full mode. Collect maintains that invariant under gThreadStateChangeLock: at the start of a generational collection it takes every allocator's chunk, and at the end, if the next mode is generational, it gives every registered allocator a new one. Collect also dereferences the chunk of every registered allocator without a null check. AttachThread, however, read sGcMode without the lock, allocated a chunk only if the mode was generational, and only then registered the allocator in AddLocal (which takes the lock). A thread that attached while a collection was running - or a pooled allocator being reattached - could read the mode before the collector changed it from full to generational, register with a null chunk after the collector had handed out chunks to the allocators registered at that time, and the next collection would dereference the null chunk. Thread-heavy programs register threads while collections run all the time. Move the decision into AddLocal, under the same lock the collector holds when it changes the mode and the chunks, so an allocator is registered with a chunk if and only if the mode is generational at that moment. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Va4FXGUp4C1CTLzsj6xMjY --- Changes.md | 1 + src/hx/gc/Immix.cpp | 19 ++++++++++++++----- 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/Changes.md b/Changes.md index 8755da7fa..0fe1886fd 100644 --- a/Changes.md +++ b/Changes.md @@ -27,6 +27,7 @@ * Fixed the moving GC updating the object id map through an iterator that the insert of the new address could have invalidated * Fixed __hxcpp_id_obj reading one past the end of the id table for an id equal to its size * Fixed the large allocation collection threshold overflowing once the block heap passes 2GB with HXCPP_GC_BIG_BLOCKS +* Fixed a crash in the next collection when a thread registered with the GC while a collection switched the generational collector on * Fixed pthread structured being unaligned * Fixed cppia crash on functions with empty bodies * Fixed regression parsing integers which wrap around diff --git a/src/hx/gc/Immix.cpp b/src/hx/gc/Immix.cpp index 407fa7dda..65846036a 100644 --- a/src/hx/gc/Immix.cpp +++ b/src/hx/gc/Immix.cpp @@ -411,6 +411,7 @@ static void VisitLocalAlloc(LocalAllocator *inAlloc,hx::VisitContext *__inCtx); static void WaitForSafe(LocalAllocator *inAlloc); static void ReleaseFromSafe(LocalAllocator *inAlloc); static void ClearPooledAlloc(LocalAllocator *inAlloc); +static void InitReferrersLocked(LocalAllocator *inAlloc); static void CollectFromThisThread(bool inMajor,bool inForceCompact); namespace hx @@ -3145,6 +3146,10 @@ class GlobalAllocator // Until we add ourselves, the collector will not wait // on us - ie, we are assumed ot be in a GC free zone. std::lock_guard lock(*gThreadStateChangeLock); + // The collector switches the gc mode and hands out or collects the referrer chunks of + // the registered allocators under this lock, so decide here, under the same lock, + // whether the new allocator needs one + InitReferrersLocked(inAlloc); mLocalAllocs.push(inAlloc); // TODO Attach debugger } @@ -5865,11 +5870,7 @@ class LocalAllocator : public hx::StackContext { GCLOG("Uncleaned referrers\n"); } - - if (sGcMode==gcmGenerational) - mOldReferrers = hx::sGlobalChunks.alloc(); - else - mOldReferrers = 0; + // mOldReferrers is assigned by AddLocal, under the thread state lock #endif @@ -6505,6 +6506,14 @@ void ClearPooledAlloc(LocalAllocator *inAlloc) inAlloc->Reset(); } +// Called with gThreadStateChangeLock held, as the allocator is registered +void InitReferrersLocked(LocalAllocator *inAlloc) +{ + #ifdef HXCPP_GC_GENERATIONAL + inAlloc->mOldReferrers = sGcMode==gcmGenerational ? hx::sGlobalChunks.alloc() : 0; + #endif +} + #ifdef HXCPP_VISIT_ALLOCS void VisitLocalAlloc(LocalAllocator *inAlloc,hx::VisitContext *__inCtx) { From edea2ae811be5ab611c6d806fb1fe0f63b822fe8 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 5 Nov 2026 06:33:38 +0000 Subject: [PATCH 8/9] Reserve one of the new blocks for the thread that grew the heap GetFreeBlock takes gThreadStateChangeLock when the free list is empty, collects if necessary, and as a last resort calls AllocMoreBlocks to add a group of 32 blocks, after which it calls GetNextFree again. GetNextFree is lock-free and the other mutator threads do not need the thread state lock to run it: the ones the collection paused are released from inside Collect, before it returns, and the ones in GC-free zones were never paused. All of them race the growing thread for the blocks it just added, and with enough threads (TestGCThreaded runs 100) they can claim all 32 before it gets there. The growing thread then falls through to "Memory exhausted" and DebuggerTrap, although memory was just added - which matches the intermittent "Memory exhausted" / illegal-instruction and trace-trap failures seen in the haxe tests (#1289). Let AllocMoreBlocks hand its first new block directly to the caller when asked, already marked owned with the zero-lock held (the state GetNextFree returns blocks in), so the thread that paid for the growth is guaranteed a block. The borrowing path used by compaction is unchanged. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Va4FXGUp4C1CTLzsj6xMjY --- Changes.md | 1 + src/hx/gc/Immix.cpp | 24 +++++++++++++++++++----- 2 files changed, 20 insertions(+), 5 deletions(-) diff --git a/Changes.md b/Changes.md index 0fe1886fd..a0d4f4398 100644 --- a/Changes.md +++ b/Changes.md @@ -28,6 +28,7 @@ * Fixed __hxcpp_id_obj reading one past the end of the id table for an id equal to its size * Fixed the large allocation collection threshold overflowing once the block heap passes 2GB with HXCPP_GC_BIG_BLOCKS * Fixed a crash in the next collection when a thread registered with the GC while a collection switched the generational collector on +* Fixed a spurious "Memory exhausted" trap when other threads claimed every block that the collecting thread had just added to the heap * Fixed pthread structured being unaligned * Fixed cppia crash on functions with empty bodies * Fixed regression parsing integers which wrap around diff --git a/src/hx/gc/Immix.cpp b/src/hx/gc/Immix.cpp index 65846036a..a18e568c9 100644 --- a/src/hx/gc/Immix.cpp +++ b/src/hx/gc/Immix.cpp @@ -3456,7 +3456,11 @@ class GlobalAllocator // malloc/new. This is not called very often, so the overhead should be minimal. // However, gcc inlines this function! requiring every alloc the have sjlj overhead. // Making it virtual prevents the overhead. - virtual bool AllocMoreBlocks(bool &outForceCompact, bool inJustBorrowing) + // + // If outReserved is given, the first new block is handed to the caller already owned and + // with its zero-lock held - the state GetNextFree returns blocks in - so that no other + // thread can claim it before the caller does. + virtual bool AllocMoreBlocks(bool &outForceCompact, bool inJustBorrowing, BlockDataInfo **outReserved = 0) { enum { newBlockCount = 1<<(IMMIX_BLOCK_GROUP_BITS) }; @@ -3541,6 +3545,13 @@ class GlobalAllocator BlockData *block = (BlockData *)(aligned + i*IMMIX_BLOCK_SIZE); BlockDataInfo *info = new BlockDataInfo(gid,block); + if (outReserved && !*outReserved) + { + info->mOwned = true; + info->mZeroLock = 1; + *outReserved = info; + } + mAllBlocks.push(info); mFreeBlocks.push(info); } @@ -3614,11 +3625,15 @@ class GlobalAllocator result = GetNextFree(inRequiredBytes); #endif + // Threads released from a collection, or that never had to stop, keep claiming free + // blocks while this one holds the thread state lock, and can take every block that + // AllocMoreBlocks adds before GetNextFree runs again. So when the heap has to grow + // on this thread's behalf, have AllocMoreBlocks reserve one of the new blocks for it + // rather than reporting the memory as exhausted when the race is lost. bool forceCompact = false; if (!result && allowMoreBlocks() && (!sgInternalEnable || GetWorkingMemory() Date: Thu, 5 Nov 2026 06:33:38 +0000 Subject: [PATCH 9/9] Report a safe point reached from inside a GC-free zone instead of hanging PauseForCollect signals mReadyForCollect and waits for mCollectDone. The collector only waits for, and only releases, threads that are not in a GC-free zone, so a thread that reaches a safe point while it is inside a zone - an allocation, GCChangeManagedMemory or __hxcpp_gc_safe_point from native code that entered the zone and forgot to leave it, or a callback into haxe made from a zone - waits for a signal that never comes and hangs for the rest of the process (#947 discusses exactly this symptom). Since mCollectDone is never left set from an earlier cycle, the wait is guaranteed to hang, so there is nothing to lose by reporting it. Raise a CriticalGCError naming the cause instead, as ExitGCFreeZone already does for a mismatched exit. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Va4FXGUp4C1CTLzsj6xMjY --- Changes.md | 1 + src/hx/gc/Immix.cpp | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/Changes.md b/Changes.md index a0d4f4398..11c182b3e 100644 --- a/Changes.md +++ b/Changes.md @@ -29,6 +29,7 @@ * Fixed the large allocation collection threshold overflowing once the block heap passes 2GB with HXCPP_GC_BIG_BLOCKS * Fixed a crash in the next collection when a thread registered with the GC while a collection switched the generational collector on * Fixed a spurious "Memory exhausted" trap when other threads claimed every block that the collecting thread had just added to the heap +* Fixed a thread hanging forever when it reached a GC safe point from inside a GC-free zone; it is now reported as a critical error (#947) * Fixed pthread structured being unaligned * Fixed cppia crash on functions with empty bodies * Fixed regression parsing integers which wrap around diff --git a/src/hx/gc/Immix.cpp b/src/hx/gc/Immix.cpp index a18e568c9..f1aa6d675 100644 --- a/src/hx/gc/Immix.cpp +++ b/src/hx/gc/Immix.cpp @@ -6083,6 +6083,11 @@ class LocalAllocator : public hx::StackContext if (sgIsCollecting) CriticalGCError("Bad Allocation while collecting - from finalizer?"); + // The collector neither waits for nor releases a thread in a GC-free zone, so waiting + // for mCollectDone from inside one would block this thread for good + if (mGCFreeZone) + CriticalGCError("PauseForCollect called from a GC-free zone - was ExitGCFreeZone missed?"); + mReadyForCollect.Set(); mCollectDone.Wait(); #endif