Repository navigation
test(master): real index builds under throttled, frozen and concurrent IO - #355
Merged
beinan merged 1 commit intoOct 8, 2026
Merged
Conversation
…t IO Review of lance-format#340 found its evidence insufficient: the long case used hand-placed checkpoint() calls rather than real IO, the real build had one row and finished instantly, and nothing asserted the build outlived the idle timeout. This adds the three scenarios asked for, all on a real BTree preparation over 64 rows: (a) Throttled: every data/index store operation sleeps 1.2 s under a 3 s idle timeout. The build takes longer than the idle timeout in total and completes, because each completed operation is progress. The test asserts the shim intercepted at least three data operations and that total time exceeded the idle timeout, so it cannot pass vacuously. (b) Frozen: the same build with storage parked is cancelled with the exact stall error within [IDLE, IDLE + 4). (c) Isolation: a healthy preparation on another table runs concurrently with (b), finishes, and neither keeps (b) alive nor is cancelled by it. Support: MergeWriteScope::with_preparation_authorizer_and_io_shim layers a test shim beneath the progress wrapper so the watchdog observes the shimmed store's completed IO exactly as it would real storage; preparation_io::test_support::ThrottleWrapper delays or freezes data-file IO. The test uses Lance's file-object-store scheme, which drives a local directory through the ObjectStore API so wrappers apply as on cloud storage (plain file paths bypass them). Mutation check: removing read-chunk and get_ranges progress accounting from ProgressStore fails (a) with "slow-but-alive real build was cancelled" after ~4 s. lance-format#340's test is kept as the cheap unit-level case. Refs lance-format#333. Review finding 6 on lance-format#340. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
17 of 18 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #333. Fixes review finding 6 on #340 (credit: production review): the earlier test used hand-placed
checkpoint()s, a one-row real build, and never asserted the build outlived the idle timeout.Scenarios (real BTree preparation, 64 rows, 3 s idle timeout)
[IDLE, IDLE+4)Support code
MergeWriteScope::with_preparation_authorizer_and_io_shim: layers a shim beneath the progress wrapper, so the watchdog counts the shimmed store's completed IO exactly as real storage.preparation_io::test_support::{Throttle, ThrottleWrapper}(#[doc(hidden)]).file-object-store://scheme: local directory through the ObjectStore API so wrappers apply as on cloud stores (plainfilepaths take the direct IO fast path and bypass them — which is why the first attempt saw 0 ops).Mutation check
Removing read-chunk +
get_rangesprogress accounting fromProgressStorefails (a) withslow-but-alive real build was cancelledafter ~4 s. Restored.Validation
New test +
slow_but_alive_…+prepared_index_task_…pass against etcd (12 s); clippy-D warnings, fmt, typos clean.🤖 Generated with Claude Code