Skip to content

MAINT GCG: make optimization iteration state explicit and typed - #2467

Open
fei (feiiiiii5) wants to merge 1 commit into
microsoft:mainfrom
feiiiiii5:issue2416/typed-optimization-run-state
Open

MAINT GCG: make optimization iteration state explicit and typed#2467
fei (feiiiiii5) wants to merge 1 commit into
microsoft:mainfrom
feiiiiii5:issue2416/typed-optimization-run-state

Conversation

@feiiiiii5

Copy link
Copy Markdown
Contributor

Purpose

Implements the structural refactor requested in #2416: a typed optimization-iteration state for the GCG loop, with the iteration lifecycle made explicit — while keeping public attack behavior and extension protocols unchanged.

Closes #2416

What changed

Typed state (attack_manager.py)

  • StopReason enum: MAX_STEPS_REACHED, ALL_PROMPTS_JAILBROKEN
  • OptimizationRunState dataclass: current suffix, best suffix, current/best loss, step counter, step runtime, stop reason. Exposed as MultiPromptAttack.last_run_state after run(), so callers and tests can programmatically inspect why/when optimization stopped
  • ProgressiveScheduleState dataclass: goals/workers admitted so far, shared step counter, loss carry-over, inner stop flag. Exposed as ProgressiveMultiPromptAttack.last_schedule_state

Explicit phases with stable contracts

  • MultiPromptAttack.run: stopping phase extracted to _all_training_prompts_jailbroken(); periodic logging phase extracted to _log_best_checkpoint() (same best-suffix swap-and-restore semantics as before, now with a finally guarantee)
  • ProgressiveMultiPromptAttack.run: scheduling counters moved into ProgressiveScheduleState; final evaluation extracted to _finalize_progressive_run()
  • GCGMultiPromptAttack.step: selection phase (flat argmin → group/in-batch decomposition) extracted to _select_best_candidate(). Candidate batches intentionally remain step-local to preserve the existing VRAM-bounding behavior noted in the loop comment

Compatibility

  • All public signatures and return values are unchanged (run() still returns (control, loss, steps) / (control, steps)); the new attributes are additive
  • No algorithm changes: acceptance/annealing, admission schedule, control-weight bumping, and logging cadence are byte-for-byte the same decisions as before, just routed through typed state

Validation

  • New tests/unit/executor/promptgen/gcg/test_run_state.py (12 tests): stop-reason assignment (incl. that the final success check does not count as an executed step), best tracking under annealing rejection, periodic checkpoint restore contract, seeded identical trajectories, argmin decomposition across worker groups, progressive finalize path with call-through assertions on the inner attack log
  • Full GCG unit suite: 211 passed (199 pre-existing + 12 new)
  • ruff check + ruff format --check clean with the repo-pinned ruff v0.16.0

AI usage disclosure

  • This PR was developed with AI assistance (Claude code agent). All code was human-reviewed, and every behavioral claim above is backed by the tests in this PR.

Human verification: I reviewed the diff hunk-by-hunk against the pre-refactor control flow, confirmed the annealing acceptance rule, admission ordering, weight bump threshold (<= 0.09), and return semantics are preserved, and re-derived the argmin index decomposition used in the selection tests.

Implements the structural half of microsoft#2416 while keeping public attack
behavior and extension protocols unchanged.

- add StopReason enum plus typed OptimizationRunState and
  ProgressiveScheduleState dataclasses capturing suffix, losses, best
  result, counters, and stop reason
- MultiPromptAttack.run now tracks state through the typed object and
  exposes it as last_run_state; stopping and periodic logging phases are
  extracted into _all_training_prompts_jailbroken and
  _log_best_checkpoint with stable contracts
- ProgressiveMultiPromptAttack.run tracks admission scheduling through
  ProgressiveScheduleState (exposed as last_schedule_state) and moves
  final evaluation into _finalize_progressive_run
- GCGMultiPromptAttack extracts the candidate-selection phase into
  _select_best_candidate; candidate batches intentionally remain
  step-local to bound VRAM
- add deterministic seeded regression tests covering stop reasons,
  best tracking under annealing rejection, checkpoint restore, argmin
  decomposition across worker groups, and progressive finalize path
@feiiiiii5
fei (feiiiiii5) force-pushed the issue2416/typed-optimization-run-state branch from 181851a to b718775 Compare August 22, 2026 20:37
@feiiiiii5

Copy link
Copy Markdown
Contributor Author

Rebased onto current main (now includes #2364 logging consolidation and #2434 ty adoption):

  • The extracted stopping/logging/finalize phases are adapted to the new module-level log helpers (_initialize_attack_log/_update_attack_log_params); no behavioral overlap remains with MAINT: Consolidate GCG attack manager logging #2364.
  • While rebasing I caught and fixed a bug in my own branch: my initial extraction had renamed the inner-run loss unpack target, which silently broke the progressive schedule's annealing continuity (prev_loss was pinned to inf across admissions instead of feeding the inner result back). The seeded regression test now asserts the correct feed-back semantics.
  • Validation on the rebased commit: GCG unit suite 217 passed (199 pre-existing incl. MAINT: Adopt ty 0.0.73 #2434's new tests + 12 new state tests), ty check clean under the repo's all = "error" config, ruff check + ruff format --check clean with the pinned ruff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MAINT Model GCG optimization iterations as explicit state

1 participant