Skip to content

fix(voice): keep emitted PCM16 audio arrays writable - #4733

Closed
subhashpolisetti wants to merge 1 commit into
openai:mainfrom
subhashpolisetti:fix/voice-writable-audio-buffer
Closed

fix(voice): keep emitted PCM16 audio arrays writable#4733
subhashpolisetti wants to merge 1 commit into
openai:mainfrom
subhashpolisetti:fix/voice-writable-audio-buffer

Conversation

@subhashpolisetti

Copy link
Copy Markdown
Contributor

Summary

StreamedAudioResult._transform_audio_buffer builds every emitted chunk with np.frombuffer, which aliases the immutable bytes it was handed, and returns that view unchanged on the int16 branch:

np_array = np.frombuffer(combined_buffer, dtype=np.int16)

if output_dtype == np.int16:
    return np_array                                          # read-only view
elif output_dtype == np.float32:
    return (np_array.astype(np.float32) / 32767.0).reshape(-1, 1)   # new, writable

The float32 branch computes a new array and is writable. TTSModelSettings.dtype defaults to np.int16, so the read-only array is what callers get unless they opt out.

Anything that edits the array in place then fails. A transform_data callback applying gain with data *= 2 raises ValueError: output array is read-only, and so does a consumer writing into VoiceStreamEventAudio.data. The identical callback succeeds when dtype is np.float32, so whether in-place work is allowed depends on a setting that says nothing about mutability, and neither the field docs nor the voice docs mention the difference.

Copying on the int16 branch is the narrowest fix. b"".join(...) currently returns the single buffered chunk unchanged, so today that branch allocates nothing; the copy adds one allocation there and leaves float32 untouched, since astype already allocates. Joining into a bytearray to get a writable view was rejected because it forces an allocation on both dtypes and makes float32 strictly more expensive for no benefit. No public signature changes.

Test plan

tests/voice/test_pipeline.py::test_voicepipeline_transform_data_can_edit_audio_in_place, parametrized over both supported dtypes. It drives the public VoicePipeline and compares the emitted samples against a baseline run of the same pipeline without the transform, so the expected values come from an independent run rather than from the implementation's own arithmetic. It asserts the baseline is non-empty first, so the comparison cannot pass vacuously.

Verified it fails without the source change by restoring src/agents/voice/result.py to main and rerunning: the int16 case fails with ValueError: output array is read-only and the float32 case passes. That split is the inconsistency the change removes.

.agents/skills/code-change-verification/scripts/run.sh passes end to end: format, lint, typecheck and the full suite. uv run mypy --platform win32 src is clean, and uv run pytest tests/voice -q is 210 passed.

Issue number

None. Found while auditing the PCM framing path in the voice pipeline.

Checks

  • I've added new tests, if relevant
  • I've run .agents/skills/code-change-verification/scripts/run.sh
  • I've confirmed all verification steps pass
  • If using Codex, I've run /review before submitting this PR

@seratch

seratch commented Aug 28, 2026

Copy link
Copy Markdown
Member

Thanks for the proposal. I don't think the SDK should make every emitted PCM16 buffer writable by copying every chunk.

transform_data already supports the use case by returning a transformed array, and a consumer that specifically needs in-place mutation can copy that array explicitly. The patch would add an allocation on the default hot path for every user without an existing writeability contract, so I am closing this.

@seratch seratch closed this Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants