Skip to content

[Tests] Add deterministic InProc Crash Reporter tests - #134228

Open
mdh1418 wants to merge 8 commits into
dotnet:mainfrom
mdh1418:feature/inproc-crashreport-android-tests
Open

mdh1418 wants to merge 8 commits into
dotnet:mainfrom
mdh1418:feature/inproc-crashreport-android-tests

Conversation

@mdh1418

@mdh1418 mdh1418 commented Sep 18, 2026

Copy link
Copy Markdown
Member

Add shared Unix and Android tests for CoreCLR's in-process crash reporter. Fixed thread, stack, exception, and register data provide stable expectations for checking report contents, output destinations, and the handling of repeated, failed, or overlapping requests.

Test structure

The tests pair a managed harness with a native library linked against CoreCLR's built PAL archive, which contains the reporter and its output components. The native driver supplies the fixed crash data through the reporter's callbacks. The managed harness invokes report generation and checks the resulting fields, frames, and files.

Signal-path tests call the reporter's dispatcher directly, allowing the process to continue so the harness can inspect its output. The test library keeps its PAL and reporter state private, so its callbacks are independent of the hosting runtime's callbacks. Each test project runs in a separate process to isolate the reporter's process-wide state.

Signal-path reporting writes JSON to a file and human-readable text to stderr on Unix or logcat on Android. The harness captures stderr or intercepts Android logging calls to validate that text alongside the JSON file. On-demand reporting delivers either format through a caller-provided callback; the test callbacks save the received bytes for the same report checks.

Test projects

The suite has matching Unix and Android projects for the following coverage:

Project Requests exercised Report checks
RichSigsegv One signal-path request JSON and console/logcat reports for SIGSEGV: three thread records, interleaved managed/native frames, generic names, exact registers and frame metadata, managed exception details, and module associations.
Abort One signal-path request JSON and console/logcat reports for SIGABRT, with a native-only crash stack and absent managed exception fields.
StackOverflow One signal-path request JSON and console/logcat output from a supplied compressed stack-overflow trace, including total frame count and repeat count.
ConsoleOnly One signal-path request with JSON file output disabled Console/logcat output, with the JSON file and report directory both absent despite a valid root being supplied.
OnDemand Sequential requests followed by controlled overlaps JSON/text callback output, rejection of nested and overlapping requests, and recovery after output failure.

On-demand checks

An on-demand caller chooses JSON or human-readable text and supplies a callback to receive the report bytes. The callback returns true to continue receiving output or false to stop. The reporter must stop calling a receiver that returns false, report that generation failed, and remain available for a later request.

OnDemand tests this contract using the rich thread and stack data from RichSigsegv. It supplies callbacks that either save all the output or deliberately return false. Sequential requests check both formats, recovery after stopped output, rejection of a missing callback, and rejection of a nested request made from inside an output callback.

The initial JSON request runs before file-output services are initialized. Later requests run with those services enabled and verify that on-demand output still goes to the caller's callback while the reporter's JSON directory remains empty.

The concurrency checks test whether another request can interfere with a report already in progress. The test pauses that report inside a callback, then makes the following requests on other threads against the same reporter:

Report already in progress Request made on another thread Expected behavior of the new request
On-demand report Another on-demand request, in JSON or text format Returns false without invoking its output callback or writing bytes.
On-demand report Signal-path request Returns without enumerating threads, writing a JSON file, or producing console/logcat output.
Signal-path report On-demand request, in JSON or text format Returns false without invoking its output callback or writing bytes.

Each new request must return before the paused report resumes. The on-demand-first checks run with both JSON and text output, and with callbacks that either accept output or stop the report when it resumes. After either outcome, fresh requests in both formats must succeed. Completed reports are checked for the expected contents.

The signal-report-first check runs last. After rejecting the overlapping requests, that report must finish with valid JSON and console/logcat output. Further on-demand requests must remain rejected because signal-triggered reporting leaves the reporter unavailable for reuse; the process would normally terminate afterwards. All thread coordination uses explicit notifications and bounded waits.

Platform and CI integration

Linux and macOS use regular CoreCLR runtime test discovery. Android packages the shared sources into functional test APKs. AndroidAppBuilder gains support for C++ extra native sources to build the test driver into those APKs.

The Android projects join the full CoreCLR suite: daily runtime-extra-platforms runs and explicitly requested runtime-android or runtime-androidemulator PR runs. They are not added to the default PR smoke selection. Windows, Mono, NativeAOT, and Apple mobile targets are excluded.

Each run writes to a unique output directory. Successful tests remove it; failures retain the files and include report excerpts in the test log. Unix Helix runs place the files in the work-item upload directory. Android failures package them into an archive that XHarness retrieves before uninstalling the app.

Validation

The project suite passed locally on Linux x64 and an Android x64 emulator without the temporary artifact probe described below. The on-demand concurrency checks passed on both platforms, and ten additional Linux runs repeated those checks successfully.

macOS, other architectures, and hosted Helix artifact collection have not been validated.

Follow-up coverage

Follow-up component tests can extend coverage of report-file cleanup, retention limits, and filesystem errors. Integration tests for actual fatal signals, runtime stack walking, concurrent fatal signals, watchdog termination, and GC-state crashes need an external controller that can inspect output after the target exits. Deterministic GC-state crashes may require GC test hooks; Apple mobile needs platform-specific launch and output collection.

Temporary CI artifact probe

The final temporary commit deliberately fails RichSigsegv after its report assertions pass and adds an incomplete-file marker. This checks that CI exposes downloadable JSON files, captured console/logcat output, and incomplete files. Revert the probe after confirming artifact collection and require passing CI before merging.

mdh1418 and others added 8 commits September 15, 2026 14:06
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: aa976a9e-b875-4524-82e1-1485d03d14fa
Link the production PAL/reporter archive, initialize a private PAL instance, and validate deterministic JSON and compact output on desktop Unix and Android.

Introduce isolated desktop runners, invocation-owned output, and bounded failure diagnostics in the shared harness.

Retain failed outputs and collect Android archives through XHarness. Use standard C++ support and route native Mono/NativeAOT exclusions before PAL dependency checks.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: aa976a9e-b875-4524-82e1-1485d03d14fa
Extend the shared fixtures and assertions with additional fatal-report shapes: native-only SIGABRT and a supplied compressed stack-overflow trace.

Add matching desktop and Android projects and document their output expectations.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: aa976a9e-b875-4524-82e1-1485d03d14fa
Reuse the rich fixture to verify compact output remains available when lifecycle file output is disabled, without creating a report directory.

Add matching desktop and Android projects and document the output expectation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: aa976a9e-b875-4524-82e1-1485d03d14fa
Reuse the shared rich assertions for repeated caller-sink reports with changing signals.

Cover null and nested requests, failed sinks and subsequent recovery, generation without services, and isolation from an enabled lifecycle sink.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: aa976a9e-b875-4524-82e1-1485d03d14fa
Hold JSON and compact-report owners in an output callback while independent threads request both formats. Require immediate rejection without callbacks, preserve owner output, and verify recovery after owner success or sink failure. Use bounded handshakes and the production reporter without product changes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: aa976a9e-b875-4524-82e1-1485d03d14fa
Exercise signal-dispatch rejection while successful and failing on-demand owners hold the shared reporter guard. Finish the existing isolated OnDemand process with a signal-shaped owner, validating output and on-demand rejection both during and after signal reporting. Reuse bounded coordination and platform capture without product hooks or real signals.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: aa976a9e-b875-4524-82e1-1485d03d14fa
Intentionally fail only RichSigsegv after its report assertions pass, before successful-run cleanup. Include an incomplete-file marker to exercise recursive retention alongside the real compact and JSON reports. Keep the expected success exit code unchanged so CI reports a genuine failure.

Revert this entire commit after confirming downloadable Linux and Android Helix artifacts, then require normal passing CI before merging.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: aa976a9e-b875-4524-82e1-1485d03d14fa
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 4 pipeline(s).
12 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The unconditional failure probe causes the RichSigsegv test and required suite to fail.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 High severity

Open (1)
What changed in this PR

Adds deterministic Unix and Android CoreCLR in-process crash reporter tests using managed/native fixtures, report validation, concurrency checks, and CI artifact handling.

Changes:

  • Adds signal, stack-overflow, console-only, abort, and on-demand scenarios.
  • Integrates native PAL-linked drivers and Android log capture.
  • Wires tests into discovery, packaging, and Helix artifact collection.
  • Includes an unconditional temporary failure probe in Program.cs that must be removed or gated.
File Summary
src/​tests/​FunctionalTests/​Android/​Device_Emulator/​InProcCrashReport/​StackOverflow/​Android.Device_Emulator.InProcCrashReport.StackOverflow.Test.csproj Android StackOverflow test adapter.
src/​tests/​FunctionalTests/​Android/​Device_Emulator/​InProcCrashReport/​Shared/​InProcCrashReport.Common.props Shared Android build configuration.
src/​tests/​FunctionalTests/​Android/​Device_Emulator/​InProcCrashReport/​Shared/​config.h Android platform definitions.
src/​tests/​FunctionalTests/​Android/​Device_Emulator/​InProcCrashReport/​Shared/​android_log_interpose.h Android log capture declarations.
src/​tests/​FunctionalTests/​Android/​Device_Emulator/​InProcCrashReport/​Shared/​android_log_interpose.c Android log interception implementation.
src/​tests/​FunctionalTests/​Android/​Device_Emulator/​InProcCrashReport/​RichSigsegv/​Android.Device_Emulator.InProcCrashReport.RichSigsegv.Test.csproj Android rich-report adapter.
src/​tests/​FunctionalTests/​Android/​Device_Emulator/​InProcCrashReport/​OnDemand/​Android.Device_Emulator.InProcCrashReport.OnDemand.Test.csproj Android on-demand adapter.
src/​tests/​FunctionalTests/​Android/​Device_Emulator/​InProcCrashReport/​ConsoleOnly/​Android.Device_Emulator.InProcCrashReport.ConsoleOnly.Test.csproj Android console-only adapter.
src/​tests/​FunctionalTests/​Android/​Device_Emulator/​InProcCrashReport/​Abort/​Android.Device_Emulator.InProcCrashReport.Abort.Test.csproj Android abort adapter.
src/​tests/​build.sh Passes runtime and build settings to CMake.
src/​tests/​baseservices/​exceptions/​inproccrashreport/​StackOverflow/​InProcCrashReport.StackOverflow.csproj Unix StackOverflow test project.
src/​tests/​baseservices/​exceptions/​inproccrashreport/​Shared/​Program.cs Managed harness and report validation; contains the unconditional failure probe.
src/​tests/​baseservices/​exceptions/​inproccrashreport/​Shared/​InProcCrashReport.Unix.props Unix test configuration.
src/​tests/​baseservices/​exceptions/​inproccrashreport/​Shared/​inproccrashreport_test_driver.cpp Synthetic native reporter driver.
src/​tests/​baseservices/​exceptions/​inproccrashreport/​Shared/​CMakeLists.txt Native test build and linking.
src/​tests/​baseservices/​exceptions/​inproccrashreport/​RichSigsegv/​InProcCrashReport.RichSigsegv.csproj Unix rich-report test project.
src/​tests/​baseservices/​exceptions/​inproccrashreport/​README.md Test scope and execution documentation.
src/​tests/​baseservices/​exceptions/​inproccrashreport/​OnDemand/​InProcCrashReport.OnDemand.csproj Unix on-demand test project.
src/​tests/​baseservices/​exceptions/​inproccrashreport/​ConsoleOnly/​InProcCrashReport.ConsoleOnly.csproj Unix console-only test project.
src/​tests/​baseservices/​exceptions/​inproccrashreport/​Abort/​InProcCrashReport.Abort.csproj Unix abort test project.
src/​tasks/​AndroidAppBuilder/​Templates/​CMakeLists-android.txt Enables C++ Android test sources.
src/​libraries/​tests.proj Includes Android CoreCLR tests.
src/​libraries/​sendtohelix-mobile.targets Configures Android failure artifact retrieval.

Comment on lines +118 to +120
// Temporary probe for Helix uploads of completed and incomplete reports.
File.WriteAllText(Path.Combine(outputDirectory, ".dotnet", "crash-reports", "incomplete.tmp"), "artifact probe\n");
Check(false, "EXPECTED_ARTIFACT_RETENTION_PROBE: assertions passed; revert this probe after verifying Helix artifacts.");
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @steveisok, @tommcdon, @dotnet/dotnet-diag
See info in area-owners.md if you want to be subscribed.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants