Conversation
Merge #7096 **Please ensure you have read the [contribution guide](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) before creating a pull request.** ### Link to Issue or Description of Change **1. Link to an existing issue (if applicable):** - Closes: #7019 - Related: #7026 (different 422: missing `RubricContent.text_property` default; does not cover this extra-fields case) **2. Or, if no issue exists, describe the change:** **Problem:** Saving an eval case from the `adk web` editor returns HTTP 422 Unprocessable Entity. The editor serializes UI-only transcript fields (`invocationIndex`, `toolUseIndex`) onto each `InvocationEvent`. `EvalBaseModel` uses `extra="forbid"`, so FastAPI rejects the PUT body as `extra_forbidden`. **Solution:** Override `InvocationEvent` with `extra="ignore"` (same ConfigDict pattern as `EvalCase` / `SessionInput`). Unknown UI fields are dropped at parse time, the save succeeds, and those indices are not persisted in the eval-case JSON. ### Testing Plan **Unit Tests:** - [x] I have added or updated unit tests for my change. - [x] All unit tests pass locally. `PYTHONPATH=src pytest tests/unittests/evaluation/test_eval_case.py` ``` 23 passed ``` Added `test_eval_case_put_accepts_web_ui_transcript_indices`, which validates the reporter payload from #7019 and asserts `invocationIndex` / `toolUseIndex` are not stored. **Manual End-to-End (E2E) Tests:** - Run `adk web` for an agent that uses tools - Create a session, add it to an eval set, edit a message, click Save - Expect HTTP 200 (not 422) - Confirm the stored eval-case JSON does not contain `invocationIndex` or `toolUseIndex` ### Checklist - [x] I have read the [CONTRIBUTING.md](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) document. - [x] I have performed a self-review of my own code. - [x] I have commented my code, particularly in hard-to-understand areas. - [x] I have added tests that prove my fix is effective or that my feature works. - [x] New and existing unit tests pass locally with my changes. - [ ] I have manually tested my changes end-to-end. - [ ] Any dependent changes have been merged and published in downstream modules. ### Additional context `#7026` only defaults `RubricContent.text_property`. The reporter confirmed that does not fix this save. Maintainer diagnosis on #7019: `extra_forbidden` on `invocationIndex` and `toolUseIndex` inside `conversation[0].intermediateData.invocationEvents`. Co-authored-by: Yi Liu <yiliuly@google.com> COPYBARA_INTEGRATE_REVIEW=#7096 from Anusha0501:fix/eval-ignore-web-ui-invocation-indices 782eb0c PiperOrigin-RevId: 982699482
9da22f0 to
df74ba7
Compare
|
Hi @i-yliu just following up on #7026 when you get a chance. This is the small RubricContent.text_property required-Optional fix discovered while investigating #7019. The later #7019 report turned out to be a separate invocationIndex/toolUseIndex issue, but this PR still fixes the independent rubric GET → PUT round-trip failure and includes regression coverage for it. I’ve kept the change focused. CI currently shows action_required; happy to address anything else needed for review/landing. Thanks! |
f00d76f to
633997e
Compare
…e without 422 RubricContent.text_property was typed Optional[str] but had no default. Under Pydantic v2 an Optional field without a default is still required, so saving an eval case whose rubric omits text_property failed model validation and the PUT returned HTTP 422 — the same required-Optional trap that google#6515 fixed for InvocationEvent.content but left unaddressed here. Default it to None so an omitted or empty text_property is accepted and round-trips through the GET(exclude_none=True) -> PUT cycle. Related google#7019
633997e to
616093c
Compare
|
Rebased onto current The bug is still live on Issue linkage. The only linked issue, #7019, is now CLOSED (2026-09-16) — it was closed by that separate #7096 fix for the other 422 (UI-only Reviewer. @surajksharma07 — you self-assigned #7019 and diagnosed the 422 here; are you the right reviewer for the eval component, or can you redirect me to whoever owns Post-rebase test run: |
Summary
Fixes a Pydantic-v2 "required Optional" trap that makes saving an eval case fail with HTTP 422 when a rubric's
text_propertyis omitted or empty.RubricContent.text_propertyis typedOptional[str]but has no default. Under Pydantic v2, anOptionalfield without a default is still required — so when the Web UI'sGETserializes an eval case withresponse_model_exclude_none=True(droppingtext_property: None) and the browser re-submits it on the savePUT, the field is missing and validation fails with422 Unprocessable Entity.This is the exact same bug class as #6336 / the #6515 fix (
fix: make InvocationEvent.content optional so the Web UI can save eval cases), which patchedInvocationEvent.contentbut leftRubricContent.text_propertyunaddressed. Related to #7019 (422 persists across 2.6.0–2.8.0 despite #6515).Root cause
src/google/adk/evaluation/eval_rubrics.py:Fix
Default
text_propertytoNone, matching the acceptedInvocationEvent.contentfix:Reproduction (before the fix)
After the fix, the round-trip succeeds and
text_propertyisNone.Testing plan
pytest tests/unittests/evaluation/test_eval_case.py— 24 passed (2 new tests:test_rubric_content_text_property_defaults_to_none,test_eval_case_with_rubric_missing_text_property_round_trips).pytest tests/unittests/evaluation/test_rubric_based_tool_use_quality_v1.py— passed (no regression in rubric consumers).Note: I ran the focused evaluation tests; the broader
tests/unittests/evaluation/suite requires optional GCS/Vertex deps not installed in the minimal venv.Signed-off-by: Aldrin Joseph yoaldrinjoseph@gmail.com