Conversation
The namespace janitor deleted a dropped table's dead `_log`/`_snap` keys one page per round, one `HEAD` + `DELETE` per key. `Gc::runNamespaceJanitor` now repeats `NamespaceJanitor::runOnePage` while a page deleted something, until a 20 s soft budget. A page deletes its dead `_log`/`_snap` keys as write-once batches split evenly across the GC I/O pool; `_ckpt` and `_files` keep the exact-token delete. After the last page of the stream the pass continues from its beginning, so debris on both sides of the cursor drains in one round. A smaller implementation of #2468. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
A pool that refused a delete job threw out of `runOnePage` and lost that page's counters and anomalies. The unscheduled keys now count as leaked, like a failed request, and the page is published. Tests: a quiet pool larger than one page takes one `LIST`; a refusal after the first job; the split across the pool on a store without a batch delete; a dead and a reborn life on one page. Docs: the phase 16 cost table and `namespaces.md` described one page per round and a `HEAD` + `DELETE` per object. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
The live namespace had no `_ckpt`, so the fold suppressed the round and the janitor page was never decided: the test passed with the `deleted > 0` clause of `more` removed. The namespace now has a checkpoint and 1200 `_files` keys, and the test asserts the published cursor. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
…tch-size setting `cas_gc_bulk_delete_chunk_keys` now also caps the namespace janitor's batch deletes; its description said only phases 15 and 17. The test pins that a listed key which is not the canonical key of its parsed identity gets no write-once form. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
- A failed `LIST` reset the persisted cursor on any page. It now resets only on the first page of a pass; later the cursor the previous page published stays, so one transient failure mid-drain does not send the next round back to the stream start. - `leaked` is zero for a page that did not publish its cursor: the page is listed again, so authority lost mid-page no longer inflates `CASGCNamespaceCleanupLeaks`. - New phase metric `delete_requests`: the delete jobs run on the GC I/O pool, so the phase row's `ProfileEvents` miss their requests. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
A batch store drains in under a tenth as many delete calls as keys; a store without a batch delete makes at least one call per key. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>
filimonov
marked this pull request as ready for review
October 5, 2026 15:40
2 tasks done
2 tasks done
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The namespace janitor (GC phase 16) deleted a dropped table's dead
_log/_snapkeys one page per folding round, with oneHEAD+DELETEper key. It now deletes a page's dead keys as write-once batches on the GC I/O pool and keeps taking pages while they delete, until a soft 20 s budget.A smaller implementation of #2468 (19 files, +858/−65 against 40 files, +3296/−729) and a replacement for it.
Changes
Gc::runNamespaceJanitorrepeatsNamespaceJanitor::runOnePagewhile a page deleted something and published its cursor. No page starts after 20 s; the page in progress finishes. A pool without debris still costs oneLISTper round.cas/ns/the pass continues from its beginning, so a pass that began mid-stream also reaches the debris before its cursor._log/_snapkeys are write-once: a page deletes them with noHEADand no token, split evenly across the GC I/O pool, at mostcas_gc_bulk_delete_chunk_keysper request. A store without a batch delete gets one request per key from the same jobs._ckptand_fileskeep the exact-token delete.LISTresets the cursor only on the first page of a pass.delete_requests,budget_exhausted.leakedis zero for a page that did not publish its cursor.Risks / notes
cas_gc_io_concurrencyrequests (16 × 63 keys by default) instead of one, and on a store without a batch delete each job first spends one refused batch call. Capping by the storage's batch-delete limit (objects_chunk_size_to_delete) is a separate PR.LISTthat lags behind deletes, a pass can keep taking pages until the budget.gc/maintenance_state: 2 extra requests per 1000 keys compared with one publication per phase.Testing
CAS*gate, debug and ASan. New suiteCASNamespaceJanitorBatches(12 tests) andCASWriteOnceKey.StreamKeyMintedFromAListedKeyMustBeThatKey; with the non-test sources reverted, the tests of the new behavior fail.tests/integration/test_cas_janitor_drainon RustFS, a disk with a batch delete and one withsupport_batch_delete = false: 2500+ dead keys drain in one deleting round.Related: #2468
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
CAS GC drains a dropped table's reference stream in one round: dead stream keys are deleted in batches on the GC I/O pool, under a soft time budget instead of one page per round.
Documentation entry for user-facing changes
docs/en/antalya/cas/architecture/garbage-collection.md,docs/en/operations/system-tables/cas_gc_log.md,docs/en/antalya/cas/configuration.md)CI/CD Options
Regression jobs to run: