fix(io): return an error for short reads - #8164
Conversation
There was a problem hiding this comment.
Gate recommendation: approve. The shared post-read validation enforces the exact physical-range contract before coalesced and split reconstruction, converting short-read panics into contextual I/O errors while covering both scheduler backends and custom readers.
| priority, | ||
| self.bypass_backpressure, | ||
| ); | ||
| let file_path = self.reader.path().clone(); |
There was a problem hiding this comment.
It might be premature optimization but it would be nice if we could avoid the path clone on every IOP. If we move the validation lower we might be able to avoid this?
There was a problem hiding this comment.
Addressed in 177c716: moved short-read validation into the physical scheduler paths, avoiding the per-I/O path clone while preserving eager lite submission and contextual I/O errors.
There was a problem hiding this comment.
✅ Gate recommendation: approve.
The revision enforces exact physical-read lengths inside both scheduler backends, eliminating the eager path clone while preserving typed errors and the existing coalescing, ordering, and backpressure contracts.
Summary
Root cause
FileSchedulerassumed every reader result exactly matched its requested physical byte range. When a reader returned fewer bytes, request reconstruction calledBytes::slicewith the expected end offset and panicked before reporting which read was incomplete.Validation
cargo fmt --allcargo test -p lance-io test_short_read_returns_io_errorcargo clippy --all --tests --benches -- -D warningscargo test -p lance-io(190 tests passed; 8 unrelated io_uring tests could not initialize in this container becauseIoUring::newreturnedEPERM)Fixes #5674