docs: add a sampling-params reference for the generation backends

[yupengtang]

What does this PR do ?

Adds docs/design-docs/sampling-params.md, a reference for how each generation backend interprets temperature / top_p / top_k.

generation.md lists them as config fields but never says how the backends read them, and they genuinely disagree — including on what "disabled" means. The page covers the two questions in #3776: how to set the params across backends, and what actually reaches each engine.

Linked from the design-docs toctree in docs/index.md and from the GenerationConfig block in docs/design-docs/generation.md.

Issues

Closes #3776.

Usage

The headline rule the page exists to state:

you write	vLLM / SGLang	TRT-LLM / Megatron
top_k: null	becomes -1 — unrestricted	becomes 0 — unrestricted
top_k: -1	unrestricted	raises — require top_k >= 0
top_k: 0	literal top-k of 0	unrestricted

So null is the portable "unrestricted" value, and examples/configs/evals/eval.yaml's top_k: -1 # -1 means disable is vLLM-shaped — it raises at request time on a TRT-LLM run. I left that config alone since it is a behaviour change rather than a docs one; happy to fix it here or in a follow-up, whichever you prefer.

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
• Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Docs-only, so no tests. python -m sphinx -b html docs/ <out> builds clean, the new page renders, and no warning references any of the three files.

Additional Information

Verification

I re-checked every row of the table in the issue against main rather than copying it, since it was written on Aug 23. All six rows still hold. Two details came out differently, and the page reflects the verified behaviour:

Greedy is not uniform across all paths. The issue says every backend uses top_k=1 for greedy. That is true of every path that takes a greedy flag — BaseVllmGenerationWorker._build_sampling_params, generate_text / generate_text_async, sglang_generation.py, trtllm_worker_async.py, megatron_worker.py, all resolving to temperature=0.0, top_k=1. But trtllm_http_server.py has no greedy parameter at all, and neither does the vLLM chat-completions handler; on the HTTP paths greedy is whatever the caller puts on the request. The page states that rather than implying a greedy flag exists there.

SGLang drops the key. It does not send top_k: -1; it omits top_k from the request dict entirely when the resolved value is -1. Worth stating since it is the one backend where "disabled" is not a value at all.

require top_k >= 0 is upstream, not in this repo. There is no such check in nemo_rl/models/generation/trtllm/. It lives in TRT-LLM's own SamplingParams (if self.top_k is not None and self.top_k < 0: raise ValueError(f"require top_k >= 0, got top_k={self.top_k}")), whose docstring also documents 0 as "all logits". tensorrt-llm==1.3.0rc21 is what pyproject.toml pins. I verified it against upstream before writing it down.

Scope

Kept to the suggested scope: the table, the null vs -1 vs 0 rule, the vLLM-HTTP top_k gap, greedy, and a pointer from generation.md. It references #2053 for where top-p/top-k support came from and #3537 for the TRT-LLM direct-vs-HTTP alignment.

I did not change any behaviour — in particular the vLLM HTTP path still ignores top_k, and the page says so explicitly rather than pretending the config applies. If you would rather that path thread top_k like TRT-LLM's now does, that is a code change and I am happy to open it separately.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.