ENH: make converter randomness reproducible and composable - #2472
Open
Roman Lutz (romanlutz) wants to merge 7 commits into
Open
ENH: make converter randomness reproducible and composable#2472Roman Lutz (romanlutz) wants to merge 7 commits into
Roman Lutz (romanlutz) wants to merge 7 commits into
Conversation
ZalgoConverter, ProportionSelectionStrategy and WordProportionSelectionStrategy called random.seed() on the process-wide RNG. Passing seed= to any one of them reset global random state on every conversion, so every other component drawing from the `random` module (~17 modules, including CharSwapConverter, RandomCapitalLettersConverter, InsertPunctuationConverter and seed sampling) silently stopped varying. Each of the three now owns a random.Random instance instead. Seeded output is byte-identical to before; only the global side effect is removed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ests Per review on microsoft#2397: - Seeding ZalgoConverter no longer implicitly seeded a randomized word selection strategy, which previously rode on the global seed. Define seeds as component-scoped and document that contract on all three seed params; seed the strategy too for end-to-end reproducibility. Adds a regression test covering both the unseeded (varies) and seeded (repeats) cases. - The new regression tests left the global RNG seeded at 0, making later tests order-dependent. Restore the original state in a finally block. Keep a setup seed distinct from the component's own seed: without it the assertion passes vacuously, since a leaking component that reseeds to the value a previous test used lands back on the captured state. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…istic-converter-outputs
Give stochastic non-LLM converters private, optional seeds and pin the documentation examples to reproducible settings. Refresh the non-LLM notebook outputs after verifying two fresh-kernel runs are identical. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 70f5e1f3-8631-48d5-9c39-5ac4f80d50d9
Configure one root seed during PyRIT initialization and derive isolated operation-local streams for stochastic converters and nested selection strategies. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 70f5e1f3-8631-48d5-9c39-5ac4f80d50d9
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 70f5e1f3-8631-48d5-9c39-5ac4f80d50d9
Preserve converter APIs and third-party behavior, derive deterministic streams from input identity, and isolate mutable generators across async tasks and worker threads. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 70f5e1f3-8631-48d5-9c39-5ac4f80d50d9
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This revises and extends #2397 so converter randomness is handled comprehensively and consistently rather than through isolated component seeds.
initialize_pyrit_async(seed=...)LLM responses remain outside this reproducibility contract.
Validation