Skip to content

perf(sampler): set max_logprobs to 0 - #461

Merged
hodaaaaaaaaaa merged 1 commit into
openrl-dev:feat/tpufrom
hodaaaaaaaaaa:perf/tpu-sampler-max-logprobs
Nov 19, 2026
Merged

hodaaaaaaaaaa merged 1 commit into
openrl-dev:feat/tpufrom
hodaaaaaaaaaa:perf/tpu-sampler-max-logprobs

Conversation

@hodaaaaaaaaaa

@hodaaaaaaaaaa hodaaaaaaaaaa commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Every OpenRL sample request asks vLLM for the sampled token's logprob and no
top-k alternatives (logprobs=0). vLLM's default max_logprobs is 20, and
with any value above 0 the vllm-torchtpu runner computes a top-20 over the
whole vocabulary on every decode step, whether or not a request asked for it.
On a v6e chip that is about 10% of a decode step at batch 64 and 128.

Change

The sampler's engine args set max_logprobs=0 on every platform, for both
LoRA and FFT samplers. The sampled token's logprob is still returned. On GPU
this changes no work: vLLM's own runner sizes the top-k from what the requests
ask for and uses max_logprobs only as a cap on that.

@hodaaaaaaaaaa hodaaaaaaaaaa changed the title perf(sampler): set max_logprobs to 0 on the TPU platform perf(sampler): set max_logprobs to 0 Nov 19, 2026
@hodaaaaaaaaaa
hodaaaaaaaaaa requested a review from ShubyM November 19, 2026 18:30
@hodaaaaaaaaaa
hodaaaaaaaaaa force-pushed the perf/tpu-sampler-max-logprobs branch from 7ecd423 to fb29931 Compare November 19, 2026 18:31
Signed-off-by: Hoda Mohammadi <hodamo@google.com>
@hodaaaaaaaaaa
hodaaaaaaaaaa force-pushed the perf/tpu-sampler-max-logprobs branch from fb29931 to 5c56303 Compare November 19, 2026 18:32
@hodaaaaaaaaaa
hodaaaaaaaaaa merged commit 77c5724 into openrl-dev:feat/tpu Nov 19, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants