DOC-6873 Fix slow Hugo builds#3686
Merged
Merged
Conversation
…ce dedup A full build had regressed to 10+ minutes. tabs/source.html was re-running Chroma over the entire source file on every slice, every step, and every page (3000+ uncached highlights); moving that behind partialCached keyed on file+language+options cuts syntax-highlight CPU by ~27% and is byte-neutral on every stable page. Chasing an apparent output change from that cache led to the real prize: the full-source template embeds on the streams page were already nondeterministic on main, shipping 8 or 0 at random. The cache was not the cause. The dedup that emits each full file once per page tracks its "seen" set in .Page.Store, which persists across a page's output-format renders; Hugo's internal RSS renders each item's full .Content, re-executing the page's shortcodes in a second render that shares that store and races the HTML render. Whichever ran first claimed the emissions, so the served HTML sometimes got none. The caching change only perturbed timing, which is why it first looked causal. The fix is a custom RSS template that builds item descriptions from front matter instead of rendering .Content, so content shortcodes never run during RSS. Emission is now deterministic for every page, not just streams. Learned: why the build was slow (uncached full-file Chroma) and why streams' full-source embeds were flaky (RSS shares .Page.Store with the HTML render) Constraint: the RSS template must not render page .Content/.Summary; doing so re-executes content shortcodes in a second output-format render that shares .Page.Store and races the HTML render, intermittently dropping full-source templates Rejected: per-output-format .Page.Store scoping | Hugo does not expose the current output format to shortcodes during content rendering Rejected: ordinal-reset or stateless first-step dedup | unsafe under concurrent output-format renders, and a shortcode cannot know the page's other steps Directive: keep the full-file Chroma highlight behind partialCached in tabs/source.html; do not inline highlight again (it re-highlights the whole file per slice, step, and page) Gaps: wall-clock speedup unverified (builds were I/O-bound on a loaded machine, only the ~27% user-CPU drop is trustworthy); RSS confirmed as the second render but other output formats not exhaustively audited Ticket: DOC-6873 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
dwdougherty
approved these changes
Jul 21, 2026
dwdougherty
left a comment
Collaborator
There was a problem hiding this comment.
I tried to test this locally, but I kept hitting rate limits on the client library sites. Approving anyway, because it's a new world now.
Contributor
Author
|
@dwdougherty You could do |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
The Hugo build had regressed to ~10+ minutes. Investigation (
hugo --templateMetrics) traced it tolayouts/partials/tabs/source.htmlre-running Chroma over the entire source file on every slice, every step, and every page — 3,000+ uncached highlights, the single most expensive thing in the build.Two changes:
Cache the full-file highlight. New
layouts/partials/tabs/highlight-file.htmldoes the read+highlight and is called viapartialCachedkeyed on(file, language, options), so each file is highlighted once per build instead of once per slice/step/page.source.htmlnow slices the cached HTML.Fix a pre-existing nondeterminism it uncovered. The
tce-fullsrcfull-file<template>dedup intabbed-clients-example.htmltracks its "seen" set in.Page.Store, which persists across a page's output-format renders. Hugo's internal RSS renders each item's full.Content, re-executing the page's shortcodes in a second render that shares that store and races the HTML render. On the heavydevelop/data-types/streams/page this meant the served HTML intermittently shipped 8 or 0 full-source templates (i.e. the "view full file" affordance was randomly broken). Newlayouts/_default/rss.xmlsources item descriptions from front matter instead of rendering.Content, so content shortcodes never run during RSS. Emission is now deterministic for every page.RSS feeds now carry each item's front-matter
description(falling back to title) instead of the full rendered page HTML. This is the mechanism of fix #2 (and is normal/lighter for feeds), but it is an outward-facing change to feed content — flagging it explicitly.Verification
Two consecutive full builds, exit 0, no template errors:
xmllint)Wall-clock speedup is not claimed as a hard number: the benchmark machine was I/O-bound (builds swung 5–16 min dominated by system time), so only the user-CPU reduction is trustworthy.
Ticket: DOC-6873
🤖 Generated with Claude Code
Note
Medium Risk
Build output and RSS item descriptions change in user-visible ways (feeds lose full HTML bodies); HTML for examples is intended to stay byte-neutral but the RSS behaviour is an explicit outward-facing change.
Overview
Speeds up Hugo builds by caching full-file Chroma output for tabbed client examples instead of re-highlighting the same source on every slice, step, and page.
tabs/source.htmlnow pulls a single highlighted file viapartialCachedon newtabs/highlight-file.html(keyed by path, language, and options) and only slices that HTML when a line range is needed. Fixes intermittent missing “view full file” templates on pages with manyclients-exampleshortcodes.A custom
layouts/_default/rss.xmlstops rendering each item’s full.Contentin the RSS pass. Item descriptions come from front matter (descriptionor title) only, so RSS no longer re-runs those shortcodes and races HTML for.Page.Store(tceFullKeys). RSS feed text changes from full page HTML to short summaries—that’s intentional and the mechanism of the determinism fix.Reviewed by Cursor Bugbot for commit b249f9f. Bugbot is set up for automated code reviews on this repo. Configure here.