Repository navigation
watcher: continuous auto-sync re-indexing churn on Windows (real driver of #832) #841
Description
During the #833 review we found the reported 20GB RSS / 100% CPU / 600MB-per-minute growth on 65 markdown files (1.3MB) cannot be produced by a single index run — the reporter's own workaround (CBM_WORKERS=1 plus auto_index false) points at the auto-sync loop re-indexing continuously on Windows and leaking per cycle.
To investigate:
- Why does the watcher consider an unchanged directory dirty every poll on Windows (mtime granularity? path-normalization mismatch in the git-status hash? CRLF)?
- Per-cycle retention: what survives repeated index cycles (candidate: mimalloc page_reclaim_on_free=0 stranding pages of exited worker heaps — see fix: memory not released after indexing (20GB+ RSS for 5MB data) #833's one plausible line).
Reproduction should loop N watcher cycles on an unchanged fixture on windows-latest and assert RSS returns to baseline after cbm_mem_collect.
Activity
fix(index): run background auto-index + watcher re-index in the super…
DeusData commentedon Jul 8, 2026
v0.9.0 includes Windows watcher/runtime and memory-retention fixes that may change this continuous auto-sync churn path, and #832 is now awaiting a reporter retest on the same class of symptom. Could you retest the unchanged-directory watcher loop on v0.9.0 and confirm whether it still re-indexes continuously? Marking awaiting-reporter.
github-actions commentedon Jul 8, 2026
Thanks for the report! To move this forward we need a bit more so we can reproduce it ourselves:
codebase-memory-mcp --versionyou're onOnce that's here we'll pick it straight back up. Heads-up: issues left
awaiting-reporterare automatically closed after a few weeks of silence, but a comment reopens the door anytime.DeusData commentedon Jul 10, 2026
The linked reporter path has activity again, so I am removing awaiting-reporter here and keeping this in the 0.9.1-rc maintainer queue. The next step is still to verify the unchanged-directory watcher loop on Windows and separate watcher churn from any retained-memory behavior.
kevinkavi2001 commentedon Aug 11, 2026
I can reproduce a deterministic Windows freshness false-positive that may help isolate the unchanged-directory churn tracked here.
Environment
C:\Users\Uzer\AppData\Local\Programs\codebase-memory-mcp\codebase-memory-mcp.exeD:\SuperDevProReproduction
index_repository.check_index_coveragefor several exact paths.os.stat(path).st_mtime_ns % 1_000_000_000.Unchanged files with whole-second mtimes are reported correctly, while unchanged files with fractional-second mtimes are reported as changed:
export-element/codepen.js0 nsmetadata_matchcontent-scripts/content.css0 nsmetadata_matchcontent-scripts/content.js212663000 nsmetadata_changedbackground.js154049700 nsmetadata_changedmanifest.json148551100 nsmetadata_changedFor the false positives,
check_index_coveragealso returns:Reindexing does not clear the result.
This reproduced after the original clean index and again after deleting the index and performing a second clean full rebuild. The second rebuild produced 309 nodes and 763 edges with
.cbmignoreexclusions applied and its UTF-8 BOM removed. Immediately afterward, the unchangedbackground.jsandmanifest.jsonfiles were still reported asmetadata_changed.Expected behavior
Unchanged files should report
metadata_matchimmediately after a clean full index, regardless of whether their NTFS mtime includes a fractional second.Suspected cause
The exact correlation suggests a Windows timestamp-normalization mismatch: index creation and freshness comparison may be storing or comparing NTFS subsecond mtimes at different precision.
Because this project is non-git and I did not reproduce the watcher loop itself, this may be either a shared timestamp-normalization defect or a separate bug in the
check_index_coveragefreshness path.fix(watcher): stop nested non-git dirs and monorepo siblings retrigge…
DeusData commentedon Sep 19, 2026
Thank you for the controlled unchanged-file comparison and for explicitly separating it from watcher churn. Current Windows coverage checking in
src/mcp/mcp.cuses whole-secondstattimestamps, which is consistent with the precision mismatch you observed. I have not established that this causes the watcher loop. This evidence should be retained as a distinct freshness-comparison case; repeated reindexing is not a resolution when the unchanged file immediately reports changed again.DeusData commentedon Oct 19, 2026
Thank you for the unchanged-file timestamp comparison. As the September reply noted, this is distinct evidence from the watcher-loop report, and repeated reindexing should not be requested as proof of a fix. I have no newly verified resolution for either path to add today.
DeusData commentedon Oct 25, 2026
Thank you for your patience. Sorry for the delay. The unchanged-file timestamp evidence remains separate from watcher churn, and both need maintainer follow-up. I’ll follow up soon.