Repository navigation
Windows module process with test kit (CI run only) - #11
Closed
basal-alfonso[bot] wants to merge 51 commits into
Closed
basal-alfonso[bot] wants to merge 51 commits into
basal-alfonso[bot] wants to merge 51 commits into
Conversation
Windows reserves COM and LPT followed by a superscript one, two or three as well as the ASCII digits. Also write the device-name check as one matches! so it passes clippy.
The module now compiles for Windows (x86_64 and aarch64 MSVC) with no warnings under clippy -D warnings, and its tests can run. Writes: - Every directory from the volume root down to the target's parent is held without FILE_SHARE_DELETE until the rename returns, so none can be moved out of the root mid-write; the parent's location is re-checked just before the rename. - A failed flush of the temporary file stops the write before the rename. - A new file is created with no explicit security descriptor, so NTFS applies real inheritance; the CreatePrivateObjectSecurity path is gone. A replaced file keeps its DACL, protection flag included. - The root is selected by the whole path, so a write to a file root works. - Intermediate directories are opened with attribute and traverse access only; a directory target is refused as "not a regular file". - A read-only target is replaced, as renameat ignores a target's mode. Roots and opens: - Root handles are reopened with the access each use needs, so a file root can be read and a directory root listed. - A failed directory query is an error, never an empty listing. - Walk errors keep their kind across roots (IO, then NOT_FOUND, then the refusal); a volume root X:\ grants its children; a file root is opened without FILE_TRAVERSE. - The reparse check fails closed, and the temp-file cleanup removes only regular files. - Directory replies are parsed as bytes bounded by the reported length, the rename buffer is 8-byte aligned and written through raw pointers, UNICODE_STRING lengths are checked, and OwnedHandle's field is private. - remove_temp walks to the lease's own directory. Tests: link-based tests assert the link exists; the swap test uses expect_err; the DACL tests apply a DACL and compare SDDL (P on replace, ID ACEs on create); new tests cover list, subdirectory create, remove_temp, legacy temps, volume roots, mount points, the held parent, flush failure and temp-name collisions. The test hooks are thread-local. The POSIX refusal test now matches the real message. fs.rs only gates helpers Windows does not use, and shares outside/io_denial/TEMP_SEQ.
A new workspace crate that starts basal's worker under the Windows confinement; off Windows it contains no code. - Profile: one shared AppContainer profile, created or opened under a session-wide named mutex; failure is appcontainer-profile-unavailable. Userenv and User32 are loaded at run time from System32 only. - Tokens: the primary is a restricted copy of the parent's token (every access group deny-only, no privileges, NULL SID as the only restricting SID, Low), lowboxed at creation with zero capabilities. The start-up thread token is a Low same-package impersonation token derived from a never-resumed AppContainer process, set on the suspended main thread and closed before resume. - Creation: CreateProcessAsUserW with STARTUPINFOEX, suspended: LPAC security capabilities with the ALL_APPLICATION_PACKAGES opt-out, the job list, eight always-on mitigations, the child-process ban, and a handle list of exactly the three stdio pipes. Explicit sorted environment (SYSTEMROOT, windir, SYSTEMDRIVE, PATH to System32, TEMP/TMP/LOCALAPPDATA to a private read-only directory), the image directory as cwd, and a private window station and desktop. The parent's environment is never passed on. - Job: flags 0x2508, one live process, no breakaway, the caller's commit limit, UI restrictions 0xff. - Checks before resume: job-limits-mismatch, not-in-owned-job, birth-token-mismatch and initial-token-open, each killing the child. - ConfinedProcess owns process, job and pipes: kill (job, then process, exit 137), try_wait, wait, token and job read-back. - Deviation: Full only without the `deviations` feature; with it, the LPAC-only and plain controls and one variant per worker and parent check. - Tests start a GUI-subsystem test child for real. CI's Windows job runs them first with `--features deviations`. - +crt-static for x86_64-pc-windows-msvc in .cargo/config.toml.
…g TEMP path On windows-latest, kill() failed with access denied: TerminateJobObject had already begun ending the worker, so the following TerminateProcess was refused while the process was not yet signaled. A successful job kill of a worker that is a member of the job is now a successful kill. The LPAC-only control's TEMP write failed with Win32 3, not 5: the system TEMP path holds 8.3 short names (RUNNER~1). The worker's TEMP is now the canonical long path, and the control's assertion accepts any denial, since only the plain control needs to show the path is writable.
…ndows-git-built-in-campaign-slice-s4-red
… fs tests pass natively)
getentropy exists only on Unix. On Windows the system-preferred RNG supplies each journaled draw, and there is still no fallback when it fails. (cherry picked from commit a69f2b0)
The workspace test can't build until every crate compiles on Windows, so basal-host's built-ins get their own test step that reports now. (cherry picked from commit 990ce58)
…nch tests pass natively)
Retain both process-attribute payloads through attribute deletion, create suspended, verify membership in the exact kill-on-close job, then resume. Use an explicit canonical drive image and a UTF-16 CRT encoder. Give each call exclusively created private config/hooks resources and read-only NUL stdin. Reuse the fs worker's no-delete-sharing handle walk via a small PinnedDirectory seam; the guard survives every git child in the operation. Retry failed job termination and put kill/close plus bounded worker cancellation inside the thread scope. Replace handwritten ABI declarations with windows-sys. Replace the original recipe-only or vacuous Windows tests with native argv round trips, job/handle observations, pinned-path controls, live descendant death, byte/line-cap writers, exceptional cleanup watchdogs, and real helper/global-config positive and negative controls. Hook controls now perform checkout rather than log, and tag writers exceed the byte budget rather than merely truncating small output. Mac and Linux basal-host tests/clippy pass; mounted MSVC lib/tests clippy and child fixture metadata checks pass with warnings denied. Native Windows execution is pending the parent-owned push/CI gate. The shared Unix canonicalize-then-git-C path race is deliberately unchanged in this slice.
Native run 38065124338 exercised all new git tests: seven failed only because GetTempPathW's trailing backslash was passed unchanged to the fs raw-spelling validator. Normalize just that host-selected base before the unchanged reparse-refusing fs walk; repository path policy remains strict. Add a regression test for that native temp spelling, inspect the actual protected DACL of directory/hooks/config (only SYSTEM and owner rights), refuse a temp junction, and attempt config/hook replacements concurrently while their guards are live. MSVC scratch lib/tests clippy -D warnings passes. Mac/Linux gates are unchanged by these Windows-only edits. All 44 fs tests passed in the first native run; rerun Windows host tests to close the seven git failures.
Native run 38065638982 had 44 fs and 21 git passes, with one handle test failure: PeekNamedPipe returned TRUE after parent writer closure. That establishes that a writer exists somewhere, not that the tested process owns it; concurrent ordinary setup/rustc spawns can inherit the same sentinel. Microsoft CreateProcessW Remarks explicitly document this multithreaded inheritance problem, and its bInheritHandles documentation says inherited handles have the same numeric value and access rights. Keep the parent's writer alive and DuplicateHandle from the exact tested child's handle table before resume and after it starts; CompareObjectHandles distinguishes the sentinel from a reused numeric slot or OS-created object. Add a deliberately unrestricted suspended CreateProcess(TRUE) positive control requiring that same-object observation to succeed. This strengthens the child-specific no-leak claim instead of relaxing it or retrying global EOF. No product handle allowlist is weakened; a real target-child leak will fail these assertions. References: learn.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-createprocessw (Parameters/Remarks), learn.microsoft.com/en-us/windows/win32/api/handleapi/nf-handleapi-duplicatehandle (Parameters/Remarks), learn.microsoft.com/en-us/windows/win32/api/handleapi/nf-handleapi-compareobjecthandles (Return value). Also preserve the original uniform git repository-denial contract when the fs pin walk fails (including missing unapproved paths), with an existence-privacy regression test. Strengthen global isolation with a real default HOME/.gitconfig positive control as well as the explicit GIT_CONFIG_GLOBAL control, so clearing inherited environment alone cannot satisfy the test. MSVC mounted lib/tests clippy with -D warnings passes; edits are Windows-only and do not impact earlier Mac/Linux gates. Native rerun pending.
The worker builds and confines itself on Windows under basal-launch: - GUI-subsystem image (no console host); an import test checks the subsystem, the forbidden GUI/COM DLLs and that no C runtime DLL is imported. - --package-sid parsing: missing exits 70 package-sid-argument-missing, unparsable or repeated exits 64. - Startup steps 1-6 before the first frame read: revert and require ERROR_NO_TOKEN, lower the primary to Untrusted and close the handle, close the CSR ALPC port and private File handles, check the actual primary token, check single mitigation bits, check the handle table against two per-image profiles that are never combined. Each failure exits 70 with its reason token. A step-3 failure (snapshot or close) is reported as handle-not-allowed, since the spec names no token for it. - The checks are pure functions tested on every system; the readers and the sequence are Windows-only. - Thread CPU clock from GetThreadTimes; JsClock takes an injectable sample source. The existing real-clock test keeps its claim but uses budgets and burns of several scheduler ticks, because Windows advances the clock a tick (~15.6 ms) at a time. - A `deviations` feature lets a test worker act on --confinement-deviation for the three checks the parent cannot break; without it the argument is refused as unknown (exit 64). Tests that use basal-testkit are gated off Windows until it builds there, and basal-testkit becomes a non-Windows dev-dependency. Windows worker tests spawn through basal-launch. The windows-baseline job runs the worker's tests on their own, with and without the feature.
… 23 Windows git tests pass natively)
The executable-name fence requires test processes to run a ckdev- copy of the worker, never a ck- named binary. The Windows worker tests copy the built worker to ckdev-basal-worker.exe (their own helper, since basal-testkit does not build on Windows yet) and launch or inspect that. The worker-depends-on-a-subc-crate control anchors on basal-worker's Cargo.lock dependency list, which now also names windows-sys; its anchor and replacement include that entry.
… worker test passes natively, with a real confined activation)
Tests read docs, fixtures and digest test vectors byte for byte. On the Windows runner a CRLF checkout broke the sandbox page test and could change the vector digests.
…or unconfined Windows children Every Windows spawn in basal takes one process-wide lock (spawn_lock) while its child's handles are inheritable; make_inheritable takes the guard so a handle cannot be made inheritable outside it. launch() now takes it from pipe creation until the child's ends are closed. spawn_plain starts an unconfined child: stdio handles (pipe or NUL) are the only inherited-handle list entries, the child is created suspended in a fresh kill-on-close job (no breakaway), verified with IsProcessInJob, then resumed. OwnedProcess is the wrapper both paths share (kill, wait, try_wait, job); ConfinedProcess derefs to it.
Use basal-launch for all runtime Windows test-kit commands and worker spawns. Track native process exit and released ownership, sample working sets, and terminate crash parents natively. Restore worker suites and wrapper-stack example; retain only named POSIX fixtures and subcases in the Windows failure evidence. Windows rejects raw relative path components, unlike Unix canonical resolution, so fs assertions now assert the platform contract while keeping the same test names. Keep the rig mutation anchor and every mutation target name unchanged. Native Windows acceptance remains to be measured by the parent-pushed CI run.
…ugh basal-launch - WorkerProcess, WorkerKiller and KillOnDrop own a basal_launch::ConfinedProcess on Windows (Unix keeps std::process::Child); the worker is launched with the job commit limit of its profile. - is_confinement_fault replaces is_sigsys: SIGSYS on Unix, 0xc0000008 / 0xc0000005 on Windows (src/windows.rs); exit 70 and the kill code are not. flow.health publishes the count as fatal_status_deaths on Windows. - sibling_worker keeps a trailing .exe: ck-basal.exe -> ck-basal-worker.exe. - subc-os is a macOS-only dependency; module self-kill hooks use fatal::kill_self (TerminateProcess on Windows). - The git runner takes basal-launch's spawn lock from its first inheritable handle until the child's ends are closed; basal-host depends on basal-launch on Windows. A native test starts confined, plain and git children at once and checks none holds another's stdio (DuplicateHandle + CompareObjectHandles). - cortexkit-store =0.2.4 (cortexkit-lease 0.1.2): the store directory and database get an owner-only protected DACL on Windows; tests/store_dacl.rs reads it back. - retention.rs stores fs_temps names portably: raw bytes on Unix (unchanged), UTF-16LE on Windows. - POSIX-only module tests are gated: tests/pool.rs signal_crash_is_reported callers (sigsys_deaths_are_counted_in_flow_health, sigkill_deaths_are_not_confinement_violations), a_wait_deadline_stops_and_reaps_its_workers; tests/e2e.rs e2e_fixture_drop_reaps_its_child. e2e kill assertions became portable. - CI windows-baseline runs basal-module + basal-core tests and the spawn-lock test.
# Conflicts: # .github/workflows/ci.yml
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
Keep the production image free of WS2_32 imports. Socket probes alone preload the DLL under the startup token, then still run every unchanged startup check before attempting WSAStartup or sockets. Preserve raw kernel trace prefixes and require PDB-resolved user creators. Disable only the diagnostic handle-tracing database after snapshot; do not change permanent strict-handle mitigation or accept a fatal checkpoint exit. Commit the identical five-DLL import inventory measured on both Windows images in CI run 38074402001. Its now-mandatory import fence gets a kernel32-removal mutation control.
Preserve both unedited production release inventories from CI run 38075560568 and render their type/access/count/creator tables. Both fit the unchanged per-image ceiling constants at 25 handles. Keep kernel trace prefixes raw and all user creators PDB-resolved to initialization. Quote every probe, thread barrier, pool and non-vacuity result from both images, including default and deviations builds. Both mutation controls redden only their named oracle and restore to an empty diff. Remove the temporary always-on provenance step: this is a one-off measurement, not a per-run worker test. The import allowlist and live handle ceilings stay enforced. Unrelated Windows workspace builds still fail in basal-core/src/retention.rs.
Use the public basal-proto limit re-export in native spawns and the direct-launch test. Bundle SQLite for the standalone Windows rig test rather than depending on a missing sqlite3.lib. Give the Unix-only tamper result an explicit Option type so the Windows untampered variant also compiles.
…ss natively on both Windows images)
Keep Unix dotdot canonicalization assertions and assert Windows raw relative-component refusal instead, preserving the approved-repository positive control and all test names. Build relative-component fixtures with native separators. Construct Windows temporary-file leases with the same drive-path spelling as the write and its roots: expanding the runner temp directory short-name alias only in the lease made the fixture inconsistent.
The per-launch name was the process id plus a counter, so a directory left by a hard-killed process collided with a later process that got the same id (create_dir failed with error 183), and the name could be guessed and planted. It is now 128 bits from BCryptGenRandom; create_dir stays exclusive, so an existing directory is never reused. A launch test plants every old-style name this process could have used and still launches.
…de gating - Manifest fs/git roots on Windows are checked with the Windows fs built-in's own validate_raw_spelling (basal-core already depends on basal-host): drive paths like C:\x are accepted; UNC, \\?\, device, drive-relative, ADS, ./.. and reserved names are refused at manifest time; ~ and ~/... stay accepted with each component held to the same rules. Unix is unchanged. - basal-core TestRuntime removes its directory after the runtime (and its open store) has dropped; basal-module's module_drop test drops its reopened store before removing the directory. An open file blocks deletion on Windows. - dry_run's built-ins test spells its paths in the long form on Windows. - Codemode is refused on Windows by basal-core's admission (unsupported_platform): the codemode end-to-end tests are ignored there and codemode_run_is_refused_on_windows asserts the refusal. - evidence/windows-build-failures.md lists the tests gated on Windows.
… context names # Conflicts: # evidence/windows-build-failures.md
…le entry basal-rig gained a Windows-only windows-sys dependency, which moved the lockfile lines this control's edit anchors on.
The worker runs JavaScript on its main thread with a QuickJS stack budget of up to 4 MiB, but the MSVC linker reserves only 1 MiB for the main thread by default. A deep recursion overflowed the native stack and killed the worker before QuickJS reported the exhausted stack budget. build.rs now passes /STACK:8388608 to the ck-basal-worker link on windows-msvc targets, matching the 8 MiB main-thread stack on macOS. A reserve is not committed memory, so the job commit limit is unaffected. tests/windows_image.rs asserts SizeOfStackReserve >= 8 MiB alongside the existing GUI-subsystem check.
Native run 38081004388 leaves one test-kit failure: Windows rejects raw dotdot components with invalid_arguments before resolution, while Unix resolves and returns denied for the escaped target. Preserve the existing no-host-dispatch, rejected settlement and audit/privacy claims; only assert the platform-specific refusal diagnostic for that one call. No test name or catalogue anchor changes.
…ack tests pass natively)
…h basal-launch, under one spawn lock (all module and core tests pass natively)
Use separate native path components when building the isolated module image path: a slash-containing join could never match QueryFullProcessImageNameW output. Add a Windows PID positive control using the current image and an absent-image negative control, so the new process-discovery path cannot pass only by returning None.
Check the constructed image locator against a literal native spelling as well as the live snapshot. Path equality normalizes separators, while the process finder compares image strings, so an OsStr assertion is needed to catch the slash-containing join regression itself. Keep test items last for native clippy.
Author
|
Superseded by #12, which targets integration/windows: this PR's base (main) moved on and conflicts, so GitHub stopped creating CI runs for it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft to run the Windows CI job on the Windows module process slice combined with the test kit slice. Not for merge as-is.
Summary by cubic
Draft combining the Windows module-process and test-kit slices so the Windows CI job runs both; not for merge as-is.
Module process on Windows
basal-modulestarts, supervises, and kills workers throughbasal-launch, keeping the.exesuffix on the sibling worker name.fatal_status_deathson Windows.cortexkit-storebumped to 0.2.4), temp-ledger names are stored as UTF-16LE, and manifest roots are validated with the Windows fs built-in's raw-spelling rules.unsupported_platform), and store-closing test fixtures order their drops so directories can be deleted.Test kit and rig portability
basal-launch's spawn lock on Windows, with native process observation and termination; the rig's process discovery is exercised against the real module image and an absent one, and the constructed image locator is asserted against a literal native spelling.mktempscratch dirs are replaced with portably named directories, and git fixtures keepSystemRootandUSERPROFILEin the environment...path components are refused withinvalid_argumentson Windows before resolution; Unix still resolves them and denies the escaped target.basal-protolimit is re-exported, and the Unix-only tamper variant compiles.deviationsfeature.Written for commit 97e08fd. Summary will update on new commits.