Skip to content

Update libdatachannel to 0.24.6 and isolate native dependencies - #3

Merged
riccardobl merged 6 commits into
masterfrom
fix/native-0246-and-fork-corrections
Oct 27, 2026
Merged

riccardobl merged 6 commits into
masterfrom
fix/native-0246-and-fork-corrections

Conversation

@toaster0123

@toaster0123 toaster0123 commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Update the native pin to official libdatachannel v0.24.6 and use published Native Bootstrap 0.1.2. Preserve the binding's MPL-2.0 license and original attribution while correcting Maven metadata and fork URLs.

Desktop JNI embeds private static OpenSSL, secure mimalloc 2.5.2 (SHA-256-pinned) and the C++ runtime. JNI, libdatachannel message buffers, libjuice, usrsctp, C++ new/delete and out-of-line C++ runtime allocations use the same private heap. OpenSSL memory callbacks are installed before initialization; failure prevents loading. macOS builds static LLVM libc++/libc++abi 20.1.8 from hash-verified official archives. Linux and MinGW embed the toolchain's static GNU C++ runtime. MinGW CRT heap calls and import-address references resolve to private definitions at link time; no process import table is patched. The corresponding runtime notices, including Winpthreads MIT/BSD notices on MinGW, are included without changing the binding's license.

Mimalloc uses MI_SECURE=ON and MI_USE_CXX=ON to retain std::bad_alloc and nothrow semantics. Linux uses local-dynamic TLS for JVM loading. MinGW selects mimalloc's supported FLS lifecycle to avoid duplicate allocator initialization with MSVCRT; configuration verifies the actual mimalloc target definition. Only JNI entry points and the existing JVM helper are public. No process allocator hooks, DYLD interposition, malloc-zone replacement, Windows CRT redirection or allocator sidecars are enabled. Java setAllocator retains its application-buffer role; platform-owned results retain their matching release APIs. Build-directory RPATHs are disabled. The forced C preamble preserves Windows rand_s declarations used by SCTP.

Android and iOS do not compile or link mimalloc and report passthrough. iOS retains private OpenSSL in one xcframework; metadata must declare privateOpenSSL=true and allocator=platform. Verification rejects mimalloc definitions and references even when local in an iOS archive, and definitions in Android JNI. Desktop verification rejects shared C++ runtimes, shared allocator/crypto dependencies, non-JNI exports and Linux static TLS.

Preserve reliability settings and numeric ICE bind addresses. Detach library-owned JNI threads before Darwin clears HotSpot state while retaining Java-owned attachments. Windows uses an owned FLS slot instead of MinGW DLL_THREAD_DETACH destructors, avoiding a loader-lock/JVM-safepoint deadlock; tracking is unregistered after native workers stop. JNI owns its Windows DLL entry point. Package dependency notices and corresponding sources with revisions/file hashes; release publication requires clean sources. Resolve prebuilt paths relative to the Gradle project and explicitly assemble selected Mac/iOS classifiers in CI.

A dedicated Allocator diagnostics workflow instruments only desktop JNI test builds on Linux, Windows, macOS ARM64 and Intel. It uses mimalloc's tracking macros to count actual allocation/free events and validate pointer ownership during initialization, component probes, repeated bidirectional DTLS transfers (64 bytes to 64 KiB), four concurrent peer pairs and native cleanup. Separate-process negative controls must reject a JVM-owned pointer and a genuinely compiled system-malloc library. A final-binary audit rejects external C heap, allocating-formatting and C++ allocation imports. Each job then rebuilds and runs the ordinary suite without instrumentation, verifies the diagnostic JNI methods disappear and uploads JSONL counters/import audits. No global allocator hook is installed. This audit found libc++'s macOS asprintf_l fallback; its result now uses the private heap, with a high-precision numeric-formatting regression. Diagnostic scope and limits are documented in docs/native-allocator-diagnostics.md.

Six logical commits are retained, all authored and committed by Toaster.

Validation:

  • Linux Dev Box: full Gradle native build and all 35 tests passed on Java 21. Linux CI also passed source packaging. ELF imports libc, libm and the loader only; no shared C++/allocator/crypto, static TLS or build RPATH.
  • macOS ARM64 and Intel: all 35 ordinary tests passed. The final binary imports libSystem only, with no standard allocator or C++ new/delete imports.
  • Windows CI: 34 tests passed and the POSIX-permissions case was skipped. Both foreign-library load orders passed 20 full child-process lifetimes each; owned JNI threads detach and Java-owned attachments remain intact. The DLL has no standard C heap, shared C++/OpenSSL/mimalloc imports. The allocator target compiled MI_SECURE=4 and MI_WIN_INIT_USE_FLS=1.
  • Allocation probes check actual pointer ownership for JNI C, C++ scalar/array/aligned allocation, RTC message objects/payloads, long strings, streams, vectors/shared pointers, libjuice, SCTP and OpenSSL. They check the allocator's compiled secure level, separation from JVM-owned buffers and throwing/nothrow allocation failure and exception-storage ownership.
  • Collision tests load a foreign library exporting OpenSSL, mimalloc and C++ new/delete symbols before and after JNI, repeatedly transfer DTLS data both ways, close peers and confirm the foreign allocators remain independent. Windows repeats the complete child-process launch, transfer and normal exit 20 times per load order.
  • Validator regressions passed: 16 dependency/platform checks, 13 Mac parser checks, three source-packager checks and seven allocator-import audit checks. PE parsing covers GNU address/hint/name, GNU ordinal/hint/name and LLVM hint/name layouts; real GNU and LLVM dumps yield the same 287 named imports.
  • All seven ordinary CI jobs passed on head 945a116: Linux, Windows, Mac ARM64/Intel, Android ARM64, iOS build and final classifier assembly. Mobile checks cover native compilation/linking/symbol policy; no mobile app runtime test is claimed.
  • All four Allocator diagnostics CI jobs passed on head 945a116. Observed allocations: Linux 3,293,718; macOS ARM64 3,742,793; macOS Intel 3,764,139; Windows 4,971,383. All 14 report phases have zero ownership violations in each positive run. Each foreign-pointer control records the expected violation, and the compiled foreign-heap library is rejected. Diagnostic and ordinary images have no audited external heap imports; ordinary images have no diagnostic entry points. The four report artifacts were downloaded and checked against their SHA-256 digests and ZIP CRCs. Tracing covers the exercised workloads and the import audit covers named direct imports; this is not an exhaustive proof over every possible execution path.

@toaster0123
toaster0123 force-pushed the fix/native-0246-and-fork-corrections branch from 95be8b5 to d08fdfe Compare October 26, 2026 13:45
@toaster0123 toaster0123 changed the title Update native sources to 0.24.6 and correct fork integration issues Update libdatachannel to 0.24.6 and correct native integration Oct 26, 2026
@toaster0123
toaster0123 marked this pull request as ready for review October 26, 2026 14:09
@toaster0123
toaster0123 force-pushed the fix/native-0246-and-fork-corrections branch 2 times, most recently from dbced2d to 1f65912 Compare October 26, 2026 15:09
@toaster0123
toaster0123 force-pushed the fix/native-0246-and-fork-corrections branch from 1f65912 to 09f6589 Compare October 26, 2026 16:05
@toaster0123 toaster0123 changed the title Update libdatachannel to 0.24.6 and correct native integration Update libdatachannel to 0.24.6 and isolate native dependencies Oct 26, 2026
@toaster0123
toaster0123 force-pushed the fix/native-0246-and-fork-corrections branch 12 times, most recently from a721929 to 4118090 Compare October 27, 2026 01:39
@toaster0123
toaster0123 force-pushed the fix/native-0246-and-fork-corrections branch from 4118090 to a2fc4df Compare October 27, 2026 01:58
@toaster0123
toaster0123 force-pushed the fix/native-0246-and-fork-corrections branch from a2fc4df to 4a49ce0 Compare October 27, 2026 02:18
@toaster0123
toaster0123 force-pushed the fix/native-0246-and-fork-corrections branch from 33f8834 to 945a116 Compare October 27, 2026 09:19
@riccardobl
riccardobl merged commit 0a71714 into master Oct 27, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants