Goal
restore bounded CSV and TSV iterable rendering.
Background and evidence
Release-preparation review of 7d8e7d7e902ace88495c5e0ff3b9db4f4a7bf5d2 on macOS arm64. Python 3.13.15, Click 8.5.0, Typer 0.27.2.
render_records eagerly materializes CSV/TSV input with [dict(record) for record in records], then prevalidates the entire list. This contradicts docs/output-contracts.md and the function docstring, which promise row streaming without retaining the complete result set. Large generators have O(n) memory growth and an unbounded producer never emits a row.
import io
from base_cli.output import render_records
for fmt in ("csv", "tsv"):
sink = io.StringIO()
def rows():
yield {"value": "first"}
assert sink.getvalue(), "first row was not written before requesting the second"
yield {"value": "second"}
render_records(rows(), requested_format=fmt, columns=(("VALUE", "value"),), stream=sink)Both formats fail the assertion; NDJSON provides a passing streaming control. The tests named test_tsv_consumes_one_pass_iterable_without_materializing and test_large_tsv_generator_is_consumed_once only prove a single iteration of the original input. Materializing into a new list passes both tests.
The eager path was introduced to reject nested non-finite cells before touching the sink. Record-atomic validation and whole-stream atomicity need an explicit compatible contract rather than an undocumented loss of streaming.
Scope and acceptance criteria
- Restore bounded memory and write-before-next-record behavior for CSV/TSV, including redirected
text.
- Define error semantics for a bad later row; keep each serialized row valid and do not weaken strict JSON handling for nested cells.
- Replace misleading streaming tests with backpressure/first-write assertions and a bounded-memory regression.
- Reconcile docs and NaN regression expectations with the selected streaming contract.
Validation
Run output tests, strict JSON contract tests, and full validation; include a generator whose second yield requires the first row to have reached the sink.
Source references:
Non-goals
No release publication or unrelated API redesign. Preserve documented compatibility except for the defective behavior identified above.
Project fields
- Project:
base-cli
- Status:
Ready
- Priority:
P2
- Area:
CLI
- Initiative:
v1.0 Readiness
- Size:
M
- Milestone:
v1.0.0
Agent assignment
Assignee: @codeforester. Implementation may be handled through the normal issue-backed worktree and reviewed PR workflow.
Goal
restore bounded CSV and TSV iterable rendering.
Background and evidence
Release-preparation review of
7d8e7d7e902ace88495c5e0ff3b9db4f4a7bf5d2on macOS arm64. Python 3.13.15, Click 8.5.0, Typer 0.27.2.render_recordseagerly materializes CSV/TSV input with[dict(record) for record in records], then prevalidates the entire list. This contradictsdocs/output-contracts.mdand the function docstring, which promise row streaming without retaining the complete result set. Large generators have O(n) memory growth and an unbounded producer never emits a row.Both formats fail the assertion; NDJSON provides a passing streaming control. The tests named
test_tsv_consumes_one_pass_iterable_without_materializingandtest_large_tsv_generator_is_consumed_onceonly prove a single iteration of the original input. Materializing into a new list passes both tests.The eager path was introduced to reject nested non-finite cells before touching the sink. Record-atomic validation and whole-stream atomicity need an explicit compatible contract rather than an undocumented loss of streaming.
Scope and acceptance criteria
text.Validation
Run output tests, strict JSON contract tests, and full validation; include a generator whose second yield requires the first row to have reached the sink.
Source references:
Non-goals
No release publication or unrelated API redesign. Preserve documented compatibility except for the defective behavior identified above.
Project fields
base-cliReadyP2CLIv1.0 ReadinessMv1.0.0Agent assignment
Assignee: @codeforester. Implementation may be handled through the normal issue-backed worktree and reviewed PR workflow.