Memory not released after indexing: 20GB+ RSS for 5MB of indexed data

Summary

After indexing a small project (65 files, 1.3MB, 2509 nodes), codebase-memory-mcp retains 20GB+ RSS on a 32GB Windows machine. Memory grows monotonically and is never released to the OS.

Environment

OS: Windows 11, i7-12700 (20 cores), 32GB RAM
CBM Version: 0.8.1
Project: 65 Markdown files, 1.3MB, 2509 nodes

Root Cause (from source code)

mimalloc abandoned-thread arenas not purged (mem.c missing mi_option_abandoned_thread_purge)
Default worker count uses all 20 cores, each with its own mimalloc arena
DEFAULT_RAM_FRACTION=0.5 means 16GB budget, no pressure to release

Observed memory

Without fix: 20GB+ RSS, growing 600MB/min, never releases
With workaround (CBM_WORKERS=1): 13MB stable

Fix

PR #831: #831

Enable mi_option_abandoned_thread_purge in mem.c
Lower DEFAULT_RAM_FRACTION from 0.5 to 0.25
Cap initial worker count at 8 in system_info.c

Workaround

Set CBM_WORKERS=1 in MCP env config and run: codebase-memory-mcp config set auto_index false

Thanks for the Windows memory-retention report and the before/after numbers. Triage: high-priority stability/performance issue under #593, with Windows impact.

This overlaps the memory-pressure work in #752/#685 but is more specific about post-index RSS retention and mimalloc worker-thread arenas. I see #833 is open against this; review should check that memory is actually returned after indexing without just hiding the issue by reducing worker parallelism too aggressively.

fix(index): gate supervised auto-index watcher registration on auto_w…

fix(index): run background auto-index + watcher re-index in the super…

Thanks for the report! To move this forward we need a bit more so we can reproduce it ourselves:

the codebase-memory-mcp --version you're on
the exact steps or command you ran
a public repo (or a small dummy snippet) that shows the problem — please don't paste proprietary code

Once that's here we'll pick it straight back up. Heads-up: issues left awaiting-reporter are automatically closed after a few weeks of silence, but a comment reopens the door anytime.

v0.9.0 includes memory-retention and RSS accounting fixes plus the indexing supervisor work, so this memory-retention report needs a fresh pass. Could you retest on v0.9.0 with the same project shape and share peak/live RSS if it still grows? Marking awaiting-reporter.

v0.9.0 Memory Test Report

Clean install of v0.9.0 (UI variant, Windows amd64), same project structure as before.

Environment

OS: Windows 10 (amd64)
Binary: codebase-memory-mcp-ui-windows-amd64.zip (v0.9.0)
Env: CBM_DIAGNOSTICS=1, MIMALLOC_PAGE_RECLAIM_ON_FREE=1
Project: 65 files, 2602 nodes, 2601 edges (same camera-docs codebase)
Duration: 5 minutes (300s) with 1 graph query

RSS Trajectory

Uptime	RSS	Peak RSS	Committed	Peak Committed	Queries
0s	6.3 MB	6.3 MB	10.4 MB	10.4 MB	0
5s	17.5 MB	17.5 MB	35.1 MB	41.3 MB	1
60s	17.4 MB	17.5 MB	35.1 MB	41.3 MB	1
180s	17.4 MB	17.5 MB	35.1 MB	41.3 MB	1
300s	17.5 MB	17.5 MB	35.1 MB	41.3 MB	1

Key Findings

Peak RSS: 17.5 MB (during query), stable at 17.4-17.5 MB afterwards
RSS growth over 5 min: ~0 MB (flat, no upward trend)
Committed: stable at 35.1 MB after initial query
Peak committed: 41.3 MB

Comparison

Previously on v0.8.1, two codebase-memory-mcp processes consumed 22.4 GB combined with CPU at 100%, growing over hours. On v0.9.0, RSS stays at ~17 MB with zero growth over 5 minutes.

The memory issue appears to be resolved in v0.9.0. The RSS is flat and memory is released back to the OS after indexing completes.

Confirming this is still present in 0.9.0 (Windows 11, 32GB RAM) — PR #831 was closed without merging (DCO/lint check failures unrelated to the code fix itself) so the root cause it identified is still live in the current release.

Repro on a real, public, large monorepo

Tested index_repository --mode fast on the llvm/ subdirectory of the public llvm-project monorepo (commit 9a2d9c7fef47c0af321150f75d8f21db06cccb43, ~80.8k files on disk, ~8.5k after language/ignore filtering) on a 32GB Windows 11 machine, default worker count (auto-detected, i7, 20 logical cores):

Worker RSS: 31MB → 9.5GB in ~3 min, then 10.7GB and still climbing ~1GB/min when I killed it (system free RAM dropped from 15GB to 5GB in the same window — this is a real stability risk, not just an inconvenience).
This matches the "20 workers × 100–500MB mimalloc arena, never purged" root cause described in fix: memory not released after indexing (20GB+ RSS for 5MB data) #831 almost exactly, just at a larger scale (more workers × more/larger files = faster runaway).

Workaround confirmed effective at this scale

Setting CBM_WORKERS=1 before the same command on the same llvm/ tree changed the picture dramatically:

Peak RSS across three indexing attempts: ~4.4GB, ~4.7GB, ~6.3GB (vs. 10.7GB+ and climbing with default workers) — well inside the auto-detected 11.27GB budget the whole time, growth rate roughly linear and predictable instead of runaway.
System free RAM stayed in the 11–15GB range throughout instead of collapsing toward 0.

So the CBM_WORKERS=1 workaround from #831 is a solid, verified stopgap for large-repo Windows users. It doesn't get you to a successful index of llvm/ on its own though — see the hang/give-up issue in #1130, which I can also reproduce on this exact same public repo (details posted there) and which is the thing actually blocking a full index now that memory is controlled.

Would be happy to help get a rebased/DCO-signed version of #831's fix through CI if that'd unblock merging it — the diff itself looked correct in my read of src/foundation/mem.c / system_info.c.

Thanks for the current v0.9.0 LLVM measurement. It confirms that the small-repository flat-RSS result and the large-repository worker-arena growth are different regimes, and that CBM_WORKERS=1 is a useful temporary mitigation rather than a fix. Please hold off on rebasing #831 for now: we need to validate the allocator and worker-budget change against current main, Windows, and the supervisor behavior from #1130 before asking you to spend more contributor time.

v0.10.3: worker memory budget exceeded during full indexing on Windows

I retested full indexing on v0.10.3. Memory usage still exceeds the reported worker budget by a large margin, even with CBM_WORKERS=1.

Environment

codebase-memory-mcp 0.10.3, standard Windows amd64 binary
Windows 11 Home x64, build 26200
Intel i5-12600KF, 16 logical processors
32 GB RAM
Windows pagefile: 4,864 MB
Installed from the GitHub release using install.ps1

Corpus

The tested Git worktree is 34 commits ahead of the public OpenJDK 21u commit 7da13d210d4c6024083c57f0b8432fc6fee19a90. The full run discovered 62,657 files.

I have not yet reproduced this on a clean checkout of the upstream commit.

Reproduction

Default full run:

codebase-memory-mcp cli --progress index_repository --repo-path '<repo>' --mode full

Full run with one worker. The daemon must be restarted because it reads environment variables at startup:

codebase-memory-mcp daemon stop
$env:CBM_WORKERS = '1'
codebase-memory-mcp daemon start
codebase-memory-mcp cli --progress index_repository --repo-path '<repo>' --mode full

Results

Default run

The full run discovered 62,657 files.
In a separate run on the same tree, 54,778 files were discovered. The log reported workers=16 and total_mb=2852 per_worker_mb=178.
RSS crossed the reported budget: rss_mb=2930 budget_mb=2852 pct=102.
Windows/mimalloc then reported cannot commit OS memory (error: 1455) and unable to commit meta-data.
The run stopped without creating an index.

level=info msg=mem.init budget_mb=2852 total_ram_mb=32602 source=daemon_worker_cap
level=info msg=pipeline.discover files=54778 elapsed_ms=6064
level=info msg=pipeline.mode mode=parallel workers=16 files=54778
level=info msg=parallel.mem.budget total_mb=2852 per_worker_mb=178
level=warn msg=mem.pressure.warn rss_mb=2930 budget_mb=2852 pct=102
mimalloc: warning: cannot commit OS memory (error: 1455 ...)
mimalloc: unable to commit meta-data ...

Run with `CBM_WORKERS=1`

The setting took effect: the log showed workers=1 and pipeline.mode mode=sequential.
The reported worker budget was still 2,852 MB.
During the definitions pass, the worker reached approximately 20,170 MB RSS and 20,417 MB private memory.
Available physical memory fell to approximately 2,372 MB, and available commit fell to approximately 600 MB.
I stopped the worker before the machine became unstable.
Immediately afterward, available physical memory recovered to approximately 26,518 MB and available commit to approximately 24,810 MB.
No index was created.

level=info msg=mem.budget.worker_cap cap_bytes=2991291699
level=info msg=mem.init budget_mb=2852 total_ram_mb=32602 source=daemon_worker_cap
level=info msg=pipeline.discover files=62657 elapsed_ms=4153
level=info msg=pass.timing pass=semantic_manifest_hash files=62657 workers=1 elapsed_ms=16592
level=info msg=pipeline.route path=full
level=info msg=pass.done pass=structure nodes=78656 edges=75957
level=info msg=pipeline.mode mode=sequential files=62657
level=info msg=pass.start pass=definitions files=62657

Expected behavior

The worker should either stay within the reported 2.85 GB budget or stop safely when it approaches that limit.

CBM_WORKERS=1 should reduce parallel overhead, but a single sequential pass should not be able to consume approximately 20 GB and exhaust Windows commit.

Notes

This is peak memory usage during indexing, not post-index retention: the run never completes.
With one worker, memory usage can still exceed the reported worker budget by approximately 7x.
I did not repeat the full workload with CBM_DIAGNOSTICS=1 because it was already bringing the machine close to instability. The measurements above came from Windows process and system counters.
I can collect a short diagnostic trace or test a diagnostic build if useful.

@fxfxfx123

Thank you all three — @fxfxfx123 for the original reading of the mimalloc arena behaviour and a clean v0.9.0 retest showing the small-project case genuinely flat, @Khusan-Khujakeldiev for the public llvm/ measurement showing the large-repository case very much alive at ~1 GB/min, and @hetasshi for the v0.10.3 numbers that matter most.

@hetasshi's run is the important correction: ~20 GB RSS against a reported 2,852 MB budget with CBM_WORKERS=1 and mode=sequential. That rules out worker-arena multiplication as the whole story, because a single sequential pass has no business exceeding its own stated budget by roughly 7x, and the error: 1455 commit failures mean the run cannot complete at all on a 32 GB machine.

Honest status: not fixed. The budget is being reported but not enforced. Tonight's v0.10.5 is being cut for the issues that make cbm impossible to operate, and "indexing exhausts the machine and never produces an index" is one of them. Sorry for how much RAM and time this has consumed on your side.

Retested/observed on v0.10.8, Windows, 64 GB RAM. This is still capable of exhausting Windows commit.

Windows Resource-Exhaustion-Detector event 2004 recorded codebase-memory-mcp.exe as the largest virtual-memory consumer several times on 2026-08-24:

17:18:46: 25,812,156,416 bytes
17:53:59: 16,699,584,512 bytes
17:54:42: 16,923,176,960 bytes
18:01:29: 19,529,547,776 bytes
18:02:32: 21,508,403,200 bytes

The CBM daemon log selected budget_mb=32332 / memory_budget_bytes=33903435776, with auto_index=true and auto_watch=true. It also registered the broad session root D:\Workspace. Empty worker logs were created across the same three incident windows (.worker-11636.log, .worker-48652.log, .worker-14360.log), which ties the growth to indexing-worker activity rather than idle query use. Diagnostics were not enabled for those runs, so there is no retained RSS trajectory.

The machine had a manually constrained 16-64 MB pagefile, which made the resulting low-commit condition crash-prone, but does not explain a CBM process growing into the 16-25.8 GB range. Other applications have independently exhausted commit on this machine too.

Applied local containment while this remains open:

CBM_MEM_BUDGET_MB=4096
CBM_WORKERS=1
CBM_DIAGNOSTICS=1
auto_index=false
auto_watch=false

I will attach the diagnostic trajectory if the process exceeds the requested cap after all clients are restarted. No proprietary source or repository contents are included here.

Follow-up under the new containment profile (CBM_MEM_BUDGET_MB=4096, CBM_WORKERS=1, diagnostics enabled), with no indexing and auto-index/watch disabled:

I exercised 17 real MCP calls through one persistent v0.10.8 client against existing graphs, including:

list_projects, index_status
search_graph (BM25 and semantic)
search_code with full source
get_code_snippet, trace_path
check_index_coverage
get_architecture on 13k-node and 138k-node projects
bounded query_graph

The largest operation was MATCH (a)-[r:CALLS]->(b) RETURN count(r) on a 138,279-node / 462,750-edge graph. It peaked at about 992 MB RSS / 955 MB committed, then returned to about 17 MB RSS / 40 MB committed. Four identical sequential runs had the same peak and no rising settled baseline.

Experiment window:

start: 13.55 MB RSS / 37.38 MB committed
end after 17 calls: 16.72 MB RSS / 40.07 MB committed
process peak: 992.33 MB RSS / 955.23 MB committed

This does not reproduce a read-only query leak under sequential single-client use. It further narrows the prior 16-25.8 GB Windows events to indexing/worker activity (or a distinct multi-client concurrency path), rather than ordinary metadata/search/trace/Cypher use. Two calls submitted concurrently through one stdio client were serialized, so independent-client query concurrency remains untested.

Fresh v0.10.8 evidence from Windows x64 confirms that the supervisor's per-worker budget is not an enforcement boundary.

With CBM_MEM_BUDGET_MB=12288 and CBM_WORKERS=1, the daemon logged:

memory_budget_bytes=12884901888
worker_memory_budget_bytes=3221225472

Despite the reported 3 GiB worker cap, three successive indexing workers reached approximately 39.0, 39.0, and 40.3 GiB private bytes. At the system level, available memory fell to 351 MiB and committed memory reached 96.3 GiB before the indexing processes exited. No repository contents or private paths are included in these measurements.

The Windows subprocess supervisor currently creates a Job Object with JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE, but does not set a Job memory limit. The worker receives the budget through argv for internal accounting only, so Windows commit remains unconstrained if that accounting is exceeded.

PR #2030 adds JOB_OBJECT_LIMIT_JOB_MEMORY using the supervisor-calculated per-worker budget and includes a native Windows regression probe. It is intentionally a host-safety boundary, not a claim that the underlying indexing growth itself is fixed.

@experticket-admin

Thank you @experticket-admin for the fresh v0.10.8 measurements and for separating host containment from the underlying indexing growth.

A reported 3 GiB worker budget alongside roughly 39 to 40 GiB private-byte peaks is strong evidence that the current Windows supervisor budget is accounting rather than an OS enforcement boundary. PR #2030 is now routed as a high-priority Windows stability fix.

Because it changes Job Object limits and failure behavior for worker process trees, we need more time to review it carefully. No additional measurements are requested from you right now. Thank you for adding a native Windows probe instead of treating the limit as self-evident.

Merge pull request #2112 from DeusData/fix/decision-a-memory-budget-f…

Fixed on main by #2112 (merge 1d66538). An over-budget index attempt now fails the whole attempt with a named error (CBM_PIPELINE_ABORT_OVER_BUDGET) after one back-pressure-futility confirmation cycle, instead of the old open-ended soft overshoot that let the kernel OOM-kill the worker. Staging is discarded through the existing rc≠0 path, so the previously serving index is kept byte-identical and no partial graph is published; the worker exits cleanly with a named reason (never a kill, so the supervisor no longer blames innocent files). The daemon also slices the worker memory budget by the number of active jobs. Thanks for the report — closing as fixed.

fix(pipeline): fail an over-budget index attempt whole instead of sof…

fix(mcp): name the cause when an index attempt stops before publication

fix(daemon): slice the worker memory budget by active jobs, not by th…

fix(pipeline): fail an over-budget index attempt whole instead of sof…

fix(mcp): name the cause when an index attempt stops before publication

fix(daemon): slice the worker memory budget by active jobs, not by th…

Navigation Menu

Repository navigation

Memory not released after indexing: 20GB+ RSS for 5MB of indexed data #832

Summary

Environment

Root Cause (from source code)

Observed memory

Fix

Workaround

v0.9.0 Memory Test Report

Environment

RSS Trajectory

Key Findings

Comparison

Repro on a real, public, large monorepo

Workaround confirmed effective at this scale

1 remaining item

v0.10.3: worker memory budget exceeded during full indexing on Windows

Environment

Corpus

Reproduction

Results

Default run

Run with `CBM_WORKERS=1`

Expected behavior

Notes

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Memory not released after indexing: 20GB+ RSS for 5MB of indexed data #832

Description

Summary

Environment

Root Cause (from source code)

Observed memory

Fix

Workaround

Activity

DeusData commented on Jul 4, 2026

github-actions commented on Jul 8, 2026

DeusData commented on Jul 8, 2026

fxfxfx123 commented on Jul 10, 2026

v0.9.0 Memory Test Report

Environment

RSS Trajectory

Key Findings

Comparison

Khusan-Khujakeldiev commented on Jul 23, 2026

Repro on a real, public, large monorepo

Workaround confirmed effective at this scale

DeusData commented on Jul 24, 2026

1 remaining item

hetasshi commented on Aug 13, 2026

v0.10.3: worker memory budget exceeded during full indexing on Windows

Environment

Corpus

Reproduction

Results

Default run

Run with CBM_WORKERS=1

Expected behavior

Notes

DeusData commented on Aug 14, 2026

rarepops commented on Aug 24, 2026

rarepops commented on Aug 24, 2026

experticket-admin commented on Sep 3, 2026

DeusData commented on Sep 3, 2026

DeusData commented on Sep 9, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Run with `CBM_WORKERS=1`