Repository navigation
Architecture pass fixes: crashes, corruption and stalls - #474
Open
dimensionscape wants to merge 8 commits into
Open
dimensionscape wants to merge 8 commits into
dimensionscape wants to merge 8 commits into
Conversation
MoveBlocks walked each source block's start bits through a local copy, so the block kept every bit. When the blocks to move into ran out part way through one, that block was kept with the start bits of the objects it had moved out, whose headers are IMMIX_OBJECT_HAS_MOVED. A stale stack word holding one of those addresses then reached GetAllocTypeChecked, which took the header's mark byte of 0 for the previous mark one collection in 16, and MarkAllocUnchecked marked 0xfe rows from there, past the 256 bytes of the row table and into the block's first data rows. Once marked, the header looked live on every later collection. The block's own bit is cleared as each object moves, and GetAllocTypeChecked refuses a moved header outright. Test: GcMovedStartBits, under HXCPP_GC_MOVING, finds a block emptied in part and puts 407 of its moved-out addresses on the stack: the collector marked all 407 as objects before and none now, 3 of 3 runs on Windows x64. Fixes #450
Three array builtins wrote to the wrong memory on valid code: - The JIT's a[i]++ with a Float result called runFloatCrement, which takes a double * for the result, with two arguments, so it stored 8 bytes through whatever the third argument register held. It passes the result's address now. - The JIT's typed push checked the capacity and expanded before it ran its argument, so an argument that grew the same array to its capacity left the store past the buffer. The argument runs first. - The interpreter's ArraySetter took a reference to the element before running the right-hand side, so a right-hand side that grew or reallocated the array stored into the old buffer. It works on a copy of the element and finds the element after, which also stores a[a.length] = a.length as the JIT and the other targets do. Test: CppiaArrayWrites, interpreted and under the JIT. Before, the JIT corrupted the Int array and its neighbour, and the interpreter stored 1 for 42 and 43 and appended 4 for 3; both engines print the expected output now, on Windows x64. Fixes #463
#404 fixed three typed-array helpers. Many other native helpers of jitted code had the same problem, in two forms: - getFieldVariant, addVariantObject, addVariantString, setFieldVariant(V), createEnum and others caught into StackContext::exception, but the jitted code had no checkException after calling them, so it ran on past the throw. The exception was then held only by StackContext::exception, which the collector did not mark, so code that allocated before the next check could free it and crash the catch or the next call. - dynamicCatDynamic, the Variant arithmetic helpers, the obj*Inc and obj*Dec helpers, getItem to setDynamicI and the DynIndex helpers, jitGet, jitSet and jitUpdate, the typed array setters and increments, array_expand(_size), the Array<Any> stores and the string iterator did not catch at all, so the exception unwound through jitted frames, which have no unwind information, past the script's catch. Each of these helpers catches now, every call to one is followed by checkException, and the collector marks a pending exception when it scans a thread's script stack, pinned, as a stack word is. The try blocks in the templated array helpers take ArrayBuiltin.cpp past MSVC's default limit on sections in an object file, so MSVC compiles with -bigobj always, as debug and sanitizer builds already did. Test: CppiaJitHelperExceptions. Before, the JIT let toString's exception from d.s += t escape the script to the host's catch; now each case is caught in its own function, interpreted and under the JIT, and nothing runs after a throw, on Windows x64. Fixes #465
Finalizers run on the collecting thread, inside the collection, with gPauseForCollect still set. The uncontended paths of Mutex, Condition, Semaphore, Deque and Lock call __hxcpp_gc_safe_point when a collection is pending, which went to SafePoint and PauseForCollect, and that ends the process when the collection is under way. Stock hxcpp took these locks in a GC-free zone, which the collecting thread can enter. sgIsCollecting is set only once every other thread is stopped or in a zone, so a thread outside a zone that reaches SafePoint while it is set is the collecting thread, and SafePoint returns for it. Test: GcFinalizerTakesMutex, 1000 finalizers that take and try a Mutex. Before, the process ended at the first; now all but those the stack scan keeps run, 3 of 3 runs on Windows x64. A finalizer that allocates, as Deque.add does, still ends the process, as it should. Fixes #452
__trace, __hxcpp_print_string and __hxcpp_println_string wrote to stdout outside a zone. When stdout blocks (a pipe nobody reads, a slow reader, a Windows console with text selected), the writing thread blocked outside the zone, so the next collection on any thread waited for it, and every other thread for the collection. They make their text first, since nothing may allocate in a zone, and write in one, through OutputZone, which enters none on a thread the collector does not know. On Windows, _hx_std_process_stdin_close called CloseHandle outside a zone, and CloseHandle waits for a WriteFile in progress on the pipe, for as long as the child does not read. Writes to stdin are now counted apart, as PipeCall counts every pipe call for destroy, and stdin_close refuses new ones and cancels those in progress before it closes, in a zone. Test: GcStdoutBlocked, a child whose stdout fills while it collects: println and trace never collected before, and now all three modes do. ProcessStdinCloseCollects: the collection and the close finished only when the child exited before, and while it runs now, on Windows x64. Fixes #454
Haxe 4's haxe.atomic.AtomicInt is a cpp.Pointer<Int>, and its methods call the _hx_atomic_* functions, none of which polled gPauseForCollect. A thread spinning on load() waiting for a flag the main thread sets after a collection kept that collection waiting for ever. Haxe 5's hx::atomic::AtomicInt calls __hxcpp_gc_safe_point in each operation, and #119 did the same for Mutex.tryAcquire. Overloads of the _hx_atomic_* functions that take ::cpp::Pointer<int>, which is what Haxe 4 passes, now stop for a pending collection and call the volatile int * forms, which the runtime's own spinlocks keep using unchanged. The __hxcpp_atomic_object_* operations stop too. Test: AtomicSpinSafePoint, with a native watchdog. Before, the collection never finished; now it does, and the spinning thread sees the flag, on Windows x64. Fixes #455
The module constructor gives the host's interface method names slots 2 to sgNativeNameSlotCount-1, which is sgNativeNameSlotCount-2 entries, and getInterfaceSlot gave a new name interfaceSlots.size()+1, which for the first is sgNativeNameSlotCount-1: the host's last slot. A class implementing a script interface and a host interface whose method has that slot (IMap's clear, in the test host) stored both methods in one slot of its interface table. New names now count from sgNativeNameSlotCount, or from 1 as before when the host registers no names. Test: CppiaInterfaceSlots, a class implementing two script interfaces and haxe.Constraints.IMap. Before, both engines crashed; now each call reaches its own method, interpreted and under the JIT, on Windows x64. Fixes #462
The TSan job reported races that are the tests' own, not the runtime's: - GcMovedStartBits reads the headers of moved-out objects on purpose, in blocks the collector's zeroing thread may be clearing. Its header read is not instrumented now. - ProcessStdinCloseCollects closes a pipe while another thread writes to it. On Windows that is the case #454 fixed, and close waits for the write; elsewhere close does not wait, so the test only checks Windows and prints the same lines elsewhere.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The first fixes from the architecture pass in #154: the crashes, memory corruption and stalls. Each commit has a regression test that fails without its fix and passes with it on Windows x64.
GC
Mutexended the process with "Bad Allocation while collecting". The collecting thread passes the safe point now, as with stock hxcpp. Test: GcFinalizerTakesMutex.Threads
trace,Sys.printandSys.printlnblocked on a full stdout outside a GC-free zone, stopping every thread at the next collection, and on Windows so didProcess.stdin.close()while another thread wrote to the child. Tests: GcStdoutBlocked, ProcessStdinCloseCollects.AtomicIntstopped every collection. Its operations are safe points now, as Haxe 5's are. Test: AtomicSpinSafePoint.cppia
a[i]++as a Float under the JIT, a typedpushwhose argument grows the array, and the interpreter'sa[i] = rhswrote to the wrong memory. Test: CppiaArrayWrites.-bigobjalways, since the templated helpers' try blocks pass the default limit on sections. Test: CppiaJitHelperExceptions.Also run locally: gc stress under moving, always-move and generational (30 s each), the cppia suite interpreted and with the JIT, and the 59 Cppia regression tests.
Fixes #450
Fixes #452
Fixes #454
Fixes #455
Fixes #462
Fixes #463
Fixes #465