refactor: report cache usage directly

[lvhan028]

Motivation

ScheduleMetrics.total_blocks and free_blocks expose allocator-specific details to the upper metrics layer.

Having the upper layer derive cache utilization from these fields couples it to a particular allocation strategy. Each engine should instead calculate cache utilization according to its own memory model and report a backend-independent value.

Modification

• Removed total_blocks and free_blocks from ScheduleMetrics.
• Changed cache_usage to a concrete float with a default of 0.0.
• Updated the PyTorch scheduler to calculate and report cache usage directly, including handling zero GPU-cache capacity.

[Copilot]

Pull request overview

This PR refactors scheduler cache reporting to expose cache_usage directly (as a normalized 0.0–1.0 fraction) and removes the older total_blocks/free_blocks fields from schedule-metrics across TurboMind (C++/pybind) and Python.

Changes:

• Remove total_blocks/free_blocks from TurboMind ScheduleMetrics (struct, logging, and pybind exposure).
• Compute and expose cache_usage directly in the PyTorch paging scheduler, with added unit coverage including a zero-GPU-blocks edge case.
• Simplify metrics aggregation to always consume ScheduleMetrics.cache_usage (no fallback derivation).

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
tests/pytorch/paging/test_scheduler.py	Adds assertions/tests validating schedule_metrics.cache_usage, including num_gpu_blocks=0.
src/turbomind/utils/metrics.h	Removes block-count fields from ScheduleMetrics and stops printing them.
src/turbomind/python/bind.cpp	Stops exporting total_blocks/free_blocks to Python bindings.
src/turbomind/engine/engine.cc	Stops populating removed block-count fields; retains cache_usage.
lmdeploy/turbomind/turbomind.py	Stops reading removed TurboMind fields when constructing Python ScheduleMetrics.
lmdeploy/pytorch/paging/scheduler.py	Computes cache_usage directly from block-manager utilization.
lmdeploy/metrics/stats.py	Uses scheduled_metrics.cache_usage directly for gpu_cache_usage.
lmdeploy/messages.py	Updates the shared ScheduleMetrics dataclass to drop block-count fields and make cache_usage non-optional.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[ErenAta16]

The motivation is stronger than the description states, and the deleted comment is the evidence for it:

// Cache-object counts. The heterogeneous allocator has no fixed-size free-block pool.
int64_t total_blocks{};  // the number of live cache objects
int64_t free_blocks{};   // always zero for the heterogeneous allocator

with engine.cc setting m->free_blocks = 0 unconditionally. So for turbomind the pair was never a free-list at all, and the consumer's derivation

self.gpu_cache_usage = 1.0 - (scheduled_metrics.free_blocks / scheduled_metrics.total_blocks)

would have reported a flat 100% utilisation had it ever run on those values. It did not, because turbomind also set cache_usage and the is None branch skipped it, but that is a coincidence of ordering rather than a design. Removing the fields removes the trap, which is a better argument for this refactor than coupling alone.

Checked the removal is complete. ScheduleMetrics has two construction sites, pytorch/paging/scheduler.py and turbomind/turbomind.py, and both are updated here along with the C++ struct and the pybind layer. Searching the tree for remaining readers of total_blocks/free_blocks turns up only stats.py:94, which this PR deletes; everything else is unrelated local naming inside the Triton kernels and the block manager's own free list.

Worth noting the refactor also closes a latent divide-by-zero. main divides unguarded, so total_blocks == 0 raises out of the metrics path; the new producer-side computation guards it:

cache_usage = 1.0 - free_blocks / total_blocks if total_blocks else 0.0

Might be worth a line in the description, since that is a behaviour fix riding along with a refactor and it is the kind of thing that gets lost when someone bisects a metrics crash later.

One question on the default. cache_usage: float = 0.0 replaces float | None = None, which drops the ability to distinguish "not reported" from "reported as empty". A backend that forgets to set it now publishes 0.0, and 0% utilisation reads as a healthy idle cache on a dashboard rather than as missing data. Given the point of the change is to stop the upper layer inventing a value, having the unset case be indistinguishable from a real measurement seems like the one place the old sentinel was earning its keep. Keeping None for "not reported" and letting the metrics layer skip the gauge rather than publish a number would preserve that, if the exporter can express an absent sample.

Not blocking, and I may be over-reading how likely a backend is to miss it now that there are only two producers.