Skip to content

Exclude test directories from Windows Defender to speed up Windows CI#385

Draft
imnasnainaec wants to merge 2 commits into
masterfrom
perf/av-exclusion-windows-tests
Draft

Exclude test directories from Windows Defender to speed up Windows CI#385
imnasnainaec wants to merge 2 commits into
masterfrom
perf/av-exclusion-windows-tests

Conversation

@imnasnainaec

@imnasnainaec imnasnainaec commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Problem

The LibChorus test suite spawns approximately 1,822 hg.exe processes during a full Windows CI run. Each process invocation takes roughly 1.9 seconds, a non-trivial portion of which is Windows Defender AV scanning the Mercurial binary and the temporary test repository directories it touches. This contributes an estimated 57 minutes of overhead (1,822 × 1.9 s) to the Windows job.

Change

Adds a new step immediately before "Test LibChorus" that registers two non-destructive AV exclusions using Add-MpPreference:

  • $env:TEMP — covers the C:\Users\runneradmin\AppData\Local\Temp\ChorusTest-* directories where test repos are created and destroyed at high frequency.
  • hg.exe (process name) — tells Defender to skip on-access scanning of the Mercurial binary itself on every invocation.

The step is gated with if: runner.os == 'Windows' so it is a no-op on Linux/macOS runners. Add-MpPreference adds to the exclusion list rather than disabling Defender globally, keeping the runner secure.

Expected Impact

AV scanning is typically 50–70% of the per-process overhead for short-lived executables on GitHub-hosted Windows runners. Excluding the temp directory and the hg.exe process should bring each invocation meaningfully closer to bare execution time, potentially reducing the Windows CI wall-clock time by 30–50 minutes.

Devin review

https://app.devin.ai/review/sillsdev/chorus/pull/385

Security trade-off of disabling AV scanning in CI

The step disables Windows Defender real-time scanning for the entire $env:GITHUB_WORKSPACE directory and the hg.exe process. While this is a common pattern to speed up CI builds and reduce flaky test failures caused by file-locking from AV scanning, it does mean that any malicious code introduced via dependencies or supply-chain attacks during the test phase would not be caught by Defender. This is generally acceptable for CI environments since GitHub-hosted runners are ephemeral, but worth noting for security-conscious teams.


This change is Reviewable

imnasnainaec and others added 2 commits June 22, 2026 15:35
Addresses review feedback on #385:
- Step now runs immediately after Build so all test steps benefit
- ExclusionPath narrowed from $env:TEMP to $env:GITHUB_WORKSPACE;
  hg.exe process exclusion already covers scanning overhead on temp
  dirs where test repos are created

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Test Results

       8 files  ±0     333 suites  ±0   2h 28m 14s ⏱️ - 9m 27s
   991 tests ±0     935 ✔️ +1    56 💤 ±0  0  - 1 
3 154 runs  ±0  3 031 ✔️ +1  123 💤 ±0  0  - 1 

Results for commit 32efab3. ± Comparison against base commit 363c8d1.

@rmunn

rmunn commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

This strikes me as a good idea; certainly scanning and re-scanning hg.exe is a waste of time. It's slow enough to start up as it is, and Chorus calls hg.exe way too often. On Linux this can be mitigated with chg, but the chg process has enough Unix-based assumptions that porting it to Windows is non-trivial. But if we have to pay the Python startup time every time we call hg (0.7 seconds last I measured it on my own device, likely different in CI), then at least we can skip paying an extra second every time for an AV scan of a file that has already been scanned thousands of times before.

@hahn-kev

hahn-kev commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

comparing the build-and-test time for both windows versions, they look pretty much the same as the last build on master. Either this didn't fix the issue or it's not a big deal. Maybe run locally and manually disable defender?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants