docs: publish translated docs in twelve languages and the tool that maintains them - #3280
Conversation
…tains them Adds machine-translated documentation sites for Simplified Chinese, Japanese, Korean and Brazilian Portuguese under /zh-CN/, /ja/, /ko/ and /pt-BR/, generated from the English pages, which stay the only source. scripts/docs/translations.py has three commands: status, translate and stage. Pages are split at their sections and only sections whose English changed are re-translated; the rest is carried forward byte-for-byte. Heading ids come from the site renderer and are pinned into the translation, code blocks are re-imposed from the English, and each generated page records the section hashes it reflects in its own front matter. Per-language inputs live in i18n/<lang>/instructions.md and glossary.json; corrections go there, never into the generated pages. The build stages each language (English overlaid with its translations plus a short notice after each page title), takes sidebar titles from the translated pages, links the single English API reference, and builds language sites non-strictly so English-only changes never fail on translations. A manually dispatched workflow refreshes a language and opens a draft PR for its reviewers.
📚 Documentation preview
|
Drops the dispatch workflow and the issue form, the translate flags that served them (--grep, --limit, --dry-run), the reviewers field, the extra exit-code and schema-validation plumbing, and the link-resolution machinery in stage: a translation that no longer fits the current English page is simply served in English with the notice. What remains is translate, stage and status plus the build wiring.
Adds Deutsch, español, français, हिन्दी, русский, Türkçe, українська and
繁體中文 alongside the existing four, using the same language codes and
switcher labels ("de - Deutsch") as other Python documentation sites;
zh-CN and pt-BR become zh and pt (Brazilian Portuguese). Each new language
gets its instructions and glossary plus a full set of generated pages.
The language switcher now keeps you on the current page when you change
language instead of returning to the home page.
maxisbey
changed the title
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
All reported issues were addressed across 657 files
Note: This PR contains a large number of files. cubic only reviews up to 200 files per PR, so some files may not have been reviewed. cubic prioritizes the most important files to review.
Re-trigger cubic
Matches a heading's trailing attribute blocks with a pattern that reads each block in one pass; the previous form could take a very long time on a heading carrying many blocks followed by other text. Parsed results are unchanged. No-Verification-Needed: one regular expression rewritten with identical results, covered by a new unit test
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Staging lays stored sections out by their recorded hashes before anything is re-imposed, so a reordered English page can no longer pair code blocks or heading ids with the wrong section; code fences are checked and restored per section and a mismatch after carry-forward goes through the repair turns like any other finding; list items and table rows are counted per section so a shortened reply is sent back whatever language its filler is in. The client is only built when a page will actually call the model, stage clears its titles marker with the tree and stages every language in one pass, and staged pages link to the English page and the API reference relative to themselves. The language switcher keeps the fragment and query. Older generated pages drop a front-matter key the tool no longer writes.
There was a problem hiding this comment.
All reported issues were addressed across 209 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
stage no longer re-imposes the current English structure onto a stored translation: a page is served exactly as it was generated (with the outdated notice once its English has moved on) or in English when there is none, so prose can never be paired with another section's code or heading ids. The switcher entries and the language sites' API entry are emitted relative to the site being built, so a mirrored copy of the build stays self-contained; the switcher script resolves the rendered roots before pointing them at the current page. Model replies end with the English's trailing newline, glossary files of the wrong shape are rejected up front, and the contributor notes describe what actually happens to a hand-edited generated page.
A stored translation can still name a docs_src file that an English change has since renamed; language configs now build with snippet path checks off, so that block renders empty under the outdated notice instead of failing the docs build. On update jobs the carried sections are put back before any check runs, so drift in text the run discards can no longer cost repair turns.
There was a problem hiding this comment.
I re-reviewed this PR at head cf1a61f and found no new bugs; a human review remains worthwhile given the size (300 files) and the CI/deploy surface it touches.
What was reviewed this pass:
- Verified both cf1a61f fixes landed as described: language configs disable snippet
check_paths(build_config.py, mirroring the tool's own_renderer()), and update jobs now runcarry_forward()before the validation pass, so only retranslated sections can cost repair turns. - One new candidate was examined and ruled out: notice links to a hard-coded translations page breaking across language sites — refuted (the staged notices resolve per-site).
- The two declined nits (ENGLISH_PAGE placeholder substitution-by-bytes,
load_registryexclude shape) remain as documented trade-offs per the resolved threads; nothing new was found in those areas.
Extended reasoning...
Overview
This pass reviewed PR head cf1a61f, which responded to the previous round of findings: the 🔴 build-stoppage (an English-only docs_src rename aborting every language build via SnippetMissingError) and the 🟡 carry-forward validation waste. Both fixes were verified in the code — build_config.py now sets check_paths: False on pymdownx.snippets for every language config, matching the override the tool's internal renderer already used, and translate_page() assembles carried sections via carry_forward() before the first reimpose() pass so reproduction drift in closed sections can no longer burn repair turns or fail the page.
Security risks
None new in this pass. The PR adds no SDK runtime code; the tool runs locally with an operator-supplied ANTHROPIC_API_KEY and nothing in CI calls the API. The main earlier concern classes (root-anchored links leaking across pinned editions, staged-page link rewriting) were already addressed in prior commits and re-litigated in resolved threads.
Level of scrutiny
High scrutiny is still appropriate: 300 files, two CI workflows (deploy-docs, docs-preview), a new build path for thirteen sites, and large volumes of machine-generated content whose prose quality is explicitly deferred to native reviewers. That is why this is a deferral rather than an approval — the automated pass covers the mechanical tool behavior, not the translation registers/terminology the PR description itself flags as provisional.
Other factors
The author has engaged with every finding across three rounds: fixed the substantive ones (00e9565, 8ed9da1, cf1a61f) and explicitly declined two nits with reasonable rationale (the placeholder-decoration case requires the model to violate an explicit instruction and fails visibly; the languages.yml shape slip surfaces immediately in status output before any paid work). This run's finder/verifier sweep produced zero confirmed findings, and the single candidate raised (hard-coded TRANSLATIONS_DOC link breakage in the notices) was adversarially refuted. Nothing outstanding remains from the bot's side; what remains is human judgment on the translations themselves and the overall design decision to ship twelve generated sites.
2 participants
Ships the docs in twelve languages — de, es, fr, hi, ja, ko, pt (Brazilian), ru,
tr, uk, zh and zh-hant, each at
/<code>/— machine-translated from the Englishpages, plus the small tool that keeps them current. English stays
the only source; everything under
i18n/<lang>/pages/is generated. Supersedes#3231 and #3265 (same goal; this takes the review of both into account —
notably no anchor edits to the English docs, no nav-label subsystem, no
second-model gate, no state file).
scripts/docs/translations.py— three commands:status,translate --lang X [--pages …],stage [--lang X]. Pages are split at##sections; only sections whose English changed are re-translatedand everything else is carried forward byte-for-byte, so a one-line English
edit is a one-line translated diff. Heading ids come from the site renderer
itself and are pinned into the translation; code blocks are re-imposed from the
English mechanically and link targets are checked against it. Provenance (section hashes) lives
in each generated page's front matter — no side state.
i18n/<lang>/instructions.md(register, voice,typography) and
glossary.json(terms, keep-list, banned renderings), thentranslate --pages …for the affected pages; the generated pages are neverhand-edited.
short notice after each H1 — machine-translated / behind the English page /
shown in English), sidebar titles come from each translated page's own H1,
and language builds are non-strict (missing snippet files included), so an
English-only PR can't fail because of translations; a page whose English has
changed since is served exactly as generated under an "outdated" notice until
refreshed, and a page never translated is shown in English. The language sites
add about a minute to the docs build in total; the API reference is linked,
not rebuilt per language.
code - nameand keeps you on the current pagewhen you change language.
docs/translations.mdexplains all this to readers.Intentional behavior changes
None to the SDK. The docs build now also produces the twelve language sites;
anthropicis added in a non-defaulttranslatedependency group (normalinstalls don't get it).
Review notes
reviewer lined up; reading order:
i18n/general-prompt.md→i18n/pt/instructions.md+glossary.json→ a few pages underi18n/pt/pages/→docs/translations.md→ the tool.packaging, tracked separately); French spacing uses ordinary rather than
no-break spaces (the model won't emit U+00A0; a small deterministic pass could); the "API Reference" nav label stays English
on language sites; a handful of terminology consistency notes for native
reviewers are listed in
i18n/<lang>/glossary.jsonnotes as provisional.ANTHROPIC_API_KEY; nothing in CI calls the API.and ids, code blocks, link targets, inline code, admonition/tab markers and
list/table counts, per section. Placement within a section, glossary
keep-terms, block titles and anything else about the prose is left to review
of the (ordinary, human-reviewed) PR that commits regenerated pages.
AI Disclaimer