Repository navigation
Make python:3.0 the default Python protocol for lf.query. - #780
Open
copybara-service[bot] wants to merge 1 commit into
Open
copybara-service[bot] wants to merge 1 commit into
copybara-service[bot] wants to merge 1 commit into
Conversation
copybara-service
Bot
force-pushed
the
test_1001607419
branch
6 times, most recently
from
October 22, 2026 20:58
55f021f to
7ef3161
Compare
## Why
`lf.query(prompt, schema, protocol='python')` resolved to `_LfQueryPythonV2`,
which renders:
```
<preamble, including a worked `Answer(final_answer=2)` example>
REQUEST:
<request, re-indented by two spaces>
OUTPUT PYTHON TYPE:
<schema, re-indented by two spaces>
OUTPUT PYTHON OBJECT:
```
Three problems follow from that layout:
1. **It defeats prefix (KV) cache reuse.** The per-request `REQUEST` block is
emitted *before* the static schema, so any change to the request
invalidates every schema token sitting behind it. Prefix caching only hits
on a byte-identical prefix, so an agent loop re-encodes its schema on every
turn even though the schema never changes.
2. **It rewrites the caller's prompt.** Request and schema are both piped
through `| indent(2, True)`, so the model never sees the text the caller
actually wrote. That matters for whitespace-significant payloads such as
source code, diffs and tables.
3. **It is longer than it needs to be.** That same `| indent(2, True)` adds
two characters per line of schema framing, on top of a worked example and
four section headers on every call.
## What
* Adds `_LfQueryPythonV3`, registered as `python:3.0`.
* Switches `_DEFAULT_PROTOCOL_VERSIONS['python']` from `'2.0'` to `'3.0'`, so
a bare `protocol='python'` (the default) now resolves to `python:3.0`.
`python:1.0` and `python:2.0` are left exactly as they are and remain
selectable via `protocol='python:1.0'` / `protocol='python:2.0'`. Any caller
that has tuned prompts or goldens against the old layout can opt out with a
one-word change. `test_explicit_version_survives_default_switch` guards exactly
that: it asserts an explicitly versioned request still renders byte-for-byte as
before, and that the default no longer matches it.
## How
`python:3.0` is a single head-first template, used for every schema:
```
<static preamble>
OUTPUT_TYPE:
<schema, verbatim>
EXAMPLES: # emitted only when few-shot examples are supplied
<...>
CONTEXT: # emitted only when the request carries context
<...>
REQUEST:
<request, verbatim>
```
* **Head-first.** Everything that does not vary with the request -- the
preamble and `OUTPUT_TYPE` -- is emitted first, and the caller's text is
last. For a fixed schema the whole prefix up to and including `REQUEST:` is
byte-identical from call to call, so the schema becomes prefix-cacheable
along with the preamble. Under `python:2.0` only the static head was
cacheable -- its 288-character preamble plus the `REQUEST:` line, 299
characters in all -- and the schema, which is unbounded and is the part
that actually costs tokens, sat behind the caller's text and was re-encoded
on every call.
* **Verbatim.** The `| indent(2, True)` filters are gone; schema and request
are rendered exactly as supplied.
* **Constant, pinned overhead.** In the common no-examples / no-context case
the template contributes exactly 439 characters of framing, independent of
the schema and the request, so `query_prompt('what is this?', int)` is 455
characters. Both numbers are asserted, not merely stated:
`test_framing_is_constant_in_schema_size` pins the 439 and its invariance
across `int`, `list[str]` and a class schema, and `test_query_prompt` pins
the 455 next to the exact rendered string.
* **One rubric instead of a worked example.** The preamble states both output
idioms -- a bare literal for primitive and container types, a
keyword-argument constructor call for class types -- and illustrates them
with `Point`, which no user schema is expected to collide with.
## Files
* `core/structured/querying.py` -- adds `_LfQueryPythonV3` and flips the
`python` default from `2.0` to `3.0`. Those are the only two hunks;
`_LfQueryPythonV1` and `_LfQueryPythonV2` are byte-for-byte untouched.
* `core/structured/querying_test.py` -- re-goldens the default-protocol
expectations and adds `LfQueryPythonV3Test` (9 tests, one per branch of the
new template, including the per-example `context` case) plus regression
guards for the older protocols.
* `core/structured/parsing_test.py`, `core/eval/base_test.py` -- re-goldened.
Several `LMSamplingUsage` assertions in `parsing_test` and under `core/eval`
are really byte-length snapshots of the rendered prompt (the fake LM reports
`len(prompt.text)` as the prompt token count), so they move with any change to
the default template.
PiperOrigin-RevId: 1001607419
copybara-service
Bot
force-pushed
the
test_1001607419
branch
from
October 22, 2026 23:25
7ef3161 to
9626d74
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Make
python:3.0the default Python protocol forlf.query.Why
lf.query(prompt, schema, protocol='python')resolved to_LfQueryPythonV2,which renders:
Three problems follow from that layout:
REQUESTblock isemitted before the static schema, so any change to the request
invalidates every schema token sitting behind it. Prefix caching only hits
on a byte-identical prefix, so an agent loop re-encodes its schema on every
turn even though the schema never changes.
through
| indent(2, True), so the model never sees the text the calleractually wrote. That matters for whitespace-significant payloads such as
source code, diffs and tables.
| indent(2, True)addstwo characters per line of schema framing, on top of a worked example and
four section headers on every call.
What
_LfQueryPythonV3, registered aspython:3.0._DEFAULT_PROTOCOL_VERSIONS['python']from'2.0'to'3.0', soa bare
protocol='python'(the default) now resolves topython:3.0.python:1.0andpython:2.0are left exactly as they are and remainselectable via
protocol='python:1.0'/protocol='python:2.0'. Any callerthat has tuned prompts or goldens against the old layout can opt out with a
one-word change.
test_explicit_version_survives_default_switchguards exactlythat: it asserts an explicitly versioned request still renders byte-for-byte as
before, and that the default no longer matches it.
How
python:3.0is a single head-first template, used for every schema:preamble and
OUTPUT_TYPE-- is emitted first, and the caller's text islast. For a fixed schema the whole prefix up to and including
REQUEST:isbyte-identical from call to call, so the schema becomes prefix-cacheable
along with the preamble. Under
python:2.0only the static head wascacheable -- its 288-character preamble plus the
REQUEST:line, 299characters in all -- and the schema, which is unbounded and is the part
that actually costs tokens, sat behind the caller's text and was re-encoded
on every call.
| indent(2, True)filters are gone; schema and requestare rendered exactly as supplied.
the template contributes exactly 439 characters of framing, independent of
the schema and the request, so
query_prompt('what is this?', int)is 455characters. Both numbers are asserted, not merely stated:
test_framing_is_constant_in_schema_sizepins the 439 and its invarianceacross
int,list[str]and a class schema, andtest_query_promptpinsthe 455 next to the exact rendered string.
idioms -- a bare literal for primitive and container types, a
keyword-argument constructor call for class types -- and illustrates them
with
Point, which no user schema is expected to collide with.Files
core/structured/querying.py-- adds_LfQueryPythonV3and flips thepythondefault from2.0to3.0. Those are the only two hunks;_LfQueryPythonV1and_LfQueryPythonV2are byte-for-byte untouched.core/structured/querying_test.py-- re-goldens the default-protocolexpectations and adds
LfQueryPythonV3Test(9 tests, one per branch of thenew template, including the per-example
contextcase) plus regressionguards for the older protocols.
core/structured/parsing_test.py,core/eval/base_test.py-- re-goldened.Several
LMSamplingUsageassertions inparsing_testand undercore/evalare really byte-length snapshots of the rendered prompt (the fake LM reports
len(prompt.text)as the prompt token count), so they move with any change tothe default template.