Repository navigation
TEST: Separate score cancellation watchdog from attack startup - #3139
Open
Richard Lundeen (richlundeen) wants to merge 1 commit into
Open
Richard Lundeen (richlundeen) wants to merge 1 commit into
Richard Lundeen (richlundeen) wants to merge 1 commit into
Conversation
Wait for validation readiness before timing cleanup. Assert the validation read was cancelled and query stored scores with an explicit filter. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 226dfd07-b984-4cc9-a0f3-502d3c02d318
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The merge queue for #3121 failed in an unrelated score cancellation test on macOS / Python 3.13. Failed job log.
The five-second watchdog covered the whole attack, including target setup and conversation persistence. The failed stack was still in
PromptNormalizer.add_conversation_to_memory_async, readingConversationEntrythrough the aiosqlite worker queue. It had not reached score validation. The log does not show a score rollback or error-result write failure.Wait up to 30 seconds for the existing validation-ready event, then keep the five-second cleanup deadline. Use the existing non-cancelling completion helper so the watchdog does not add another cancellation to the operation under test. Assert that the held validation read received cancellation and that error propagation did not require releasing the read. Query scores with an explicit type filter; an unfiltered query returns no rows by design and cannot check rollback.
This is a test-only change. The auxiliary-scorer failure, transaction-start event, blocked validation read, exception cause, and persisted attack-error assertions remain in place. No production code or changes from #3121 are included.
Tests and Documentation
Validation on Windows / uv-managed Python 3.12.12:
uv run --no-sync pytest tests\unit\executor\attack\test_scoring_expectation_transport.py -q --durations=5— passed, 41 tests.uv run --no-sync pytest tests\unit\executor\attack\test_scoring_expectation_transport.py tests\unit\memory\test_sqlite_cancellation.py -n 4 --dist=loadfile -q— passed, 78 tests on the final diff.$env:UV_NO_SYNC='1'; uv run --no-sync pre-commit run --all-files— all hooks passed on the final diff.A temporary pytest plugin outside the repository added a 5.1-second async delay before target sending. The original test failed with
TimeoutErrorbefore scoring; the revised test passed with the same delay. The revised test does not add this delay to normal CI runs. The original macOS / Python 3.13 environment was not available locally; CI must confirm that platform.No product documentation changes apply to this test-only fix. JupyText was not run.