Repository navigation
feat: opt-in 4:4:4 chroma for SDR MP4 masters - #5820
Open
PaprikaChen wants to merge 9 commits into
Open
PaprikaChen wants to merge 9 commits into
PaprikaChen wants to merge 9 commits into
Conversation
PNG capture used to imply a cleared page background. A new CaptureOptions.transparentBackground lets lossless opaque PNG capture (4:4:4 MP4 output) keep the page background. Absent, it still follows format === "png", so alpha output and motion blur are unchanged. Opaque PNG skips drawElement, whose verification and worker-encode paths are JPEG-only. Refs heygen-com#5555 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
getEncoderPreset takes an optional chromaSubsampling ("420" by
default). "444" returns yuv444p for SDR MP4, so libx264 encodes the
High 4:4:4 Predictive profile and odd sizes need no even padding.
WebM, MOV and HDR presets ignore it. The ChromaSubsampling type is
exported for the producer.
Refs heygen-com#5555
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ture
RenderConfig.chromaSubsampling ("420" by default) opts an SDR MP4
render into 4:4:4. "444" encodes libx264 yuv444p and captures every
frame as opaque PNG: Chrome's quality-100 JPEG is already 4:2:0 on
beginFrame and screenshot capture, and drawElement can fall back to
screenshot mid-render, so PNG is the only capture that keeps the
colour. drawElement is turned off for these renders.
The probe and main capture options share resolveCaptureSurface, so
they cannot disagree on format, quality or background.
Formats other than mp4, GPU encoding and HDR output are rejected
before any work starts, auto-detected HDR sources fail before capture,
and distributed requests refuse 444. The default render is unchanged.
Refs heygen-com#5555
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shared parseEncoderOptions validates chromaSubsampling and answers 400 for "444" with a non-mp4 format, gpu, or force-hdr, so a request that would come back 4:2:0 never starts a render. The Studio render route, both Studio adapters and the producer HTTP server forward it. The producer server also refuses "444" with outputDynamicRange "hdr", which parseEncoderOptions cannot see. No Studio UI control; the option is route-level like crf and hdrMode. Refs heygen-com#5555 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`hyperframes render --chroma-subsampling 444` opts an SDR MP4 render into full-resolution chroma (yuv444p with lossless PNG capture). The default stays 420. The plan rejects 444 with a non-mp4 format, --gpu or --hdr before anything renders, with a usage error that names the conflicting flag. The value reaches the local render request and, only when it is 444, the Docker argv, so default Docker invocations are unchanged. Refs heygen-com#5555 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add a "4:4:4 masters" section to the rendering guide: when to use --chroma-subsampling 444, why it captures PNG, the measured chroma gain (large for text and UI, none for footage and gradients), its limits, and a player compatibility table to be filled in from testing. Document the flag in the CLI reference, the built-in rendering docs and the hyperframes-cli skill reference (with the regenerated skills manifest), and the chromaSubsampling option in the producer docs. Refs heygen-com#5555 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fill the 4:4:4 compatibility table from testing H.264 High 4:4:4 Predictive output. Chrome 155, Edge 155, VLC 3.0, Safari on iOS 26.6 and Safari 26.6 on macOS play it, and so does AVFoundation on macOS 26.6, the framework QuickTime Player plays through. Firefox 157 on Windows and both Windows media players do not. Refs heygen-com#5555 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace the vague "slower capture" note with measured numbers: a text and UI composition rendered in about twice the default time, full-frame footage in about three and a half times, because PNG encoding cost grows with image detail. Refs heygen-com#5555 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ling-444 # Conflicts: # packages/producer/src/renderRequest.ts # packages/producer/src/services/renderOrchestrator.ts # skills-manifest.json
Preview approval requiredThis pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds an opt-in
chromaSubsampling: "420" | "444"render option (--chroma-subsamplingonhyperframes render, aRenderConfigfield, and the Studio / producer HTTP render routes)."444"renders an SDR MP4 master with full-resolution colour: libx264yuv444p(High 4:4:4 Predictive) encoded from lossless PNG capture. The default stays"420"and the default render path is unchanged.Why
Every SDR MP4 is 4:2:0, which softens or fringes thin coloured text, UI strokes and saturated edges. Motion-graphics masters are mostly that. 4:4:4 keeps the colour, but many players cannot decode it, so it is opt-in only.
Related work
Fixes #5555. The approach was posted on the issue before implementing.
Measurement (issue step 1)
Each clip was encoded from a lossless RGB master with libx264
mediumat the same CRF, using the engine's SDR colour conversion (BT.709,aq-mode=3). PSNR and SSIM are computed per plane against the master. Windows 11, ffmpeg 9.0.2, 1920x1080, 30 fps, 5 s.CRF 23 shows the same pattern. Large gain for text and UI (about +13 to +16 dB chroma, visibly cleaner edges, smaller files at the same CRF); none for footage or gradients.
4x nearest-neighbour crops (master | 4:2:0 | 4:2:0 lanczos | 4:4:4):
End to end through this PR (
hyperframes rendervshyperframes render --chroma-subsampling 444, compared with a PNG-capture reference): chroma PSNR 21.9 / 22.9 dB vs 44.6 / 45.4 dB on a 1 px red/blue stripe test.How
"444"captures PNG on every path and turns drawElement off (compile gatechroma_444).CaptureOptions.transparentBackgroundseparates "PNG capture" from "transparent background". Absent, it followsformat === "png"as before, so alpha output and motion blur are unchanged;falsekeeps the page background so an opaque MP4 does not turn black. Opaque PNG skips drawElement (JPEG-only verification and worker encode).getEncoderPresettakes an optionalchromaSubsamplingthat only changes the SDR MP4 branch toyuv444p; x264 picks High 4:4:4 Predictive and odd sizes need no padding.RenderConfig.chromaSubsampling;resolveCaptureSurfacegives the probe and main capture the same format, quality and background; the preset gets the chroma value, so every encode path and the segment-resume hash seeyuv444p."444"with a non-mp4 format, GPU encoding or HDR output is rejected before any work starts; auto-detected HDR sources fail before capture with a--sdrhint; distributed / Lambda / Cloud Run requests refuse it (chunk workers and the CFR assemble are 4:2:0). The CLI, the Studio route (400) and the producer HTTP server reject the same combinations.yuv444p(QSV and VAAPI are NV12-only here, AMF has no 4:4:4 H.264, VideoToolbox H.264 has no 4:4:4 profile), so GPU encoding is rejected rather than silently falling back to 4:2:0 on some hosts. NVENC support could be a follow-up.Player compatibility (issue step 3)
Also in the rendering guide.
canPlayTypeemptyEvery player above plays the default 4:2:0 output.
Cost
Median of 3, 1 capture worker, 1920x1080, 300 frames, Windows 11 with a hardware GPU.
About 2x the default render time for text and UI, about 3.5x for full-frame footage: PNG encoding cost grows with image detail. Documented in the guide.
Out of scope (noted, not changed)
flags=neighbor; lanczos gains about 1.5 to 2 dB chroma on text and UI with no compatibility cost. Happy to open a separate issue.transparentBackground: falsewould fix it in one line.optimizeForSpeedoff (it protects alpha); enabling it for opaque PNG may cut the 444 capture cost.Test plan
getEncoderPreset, encoder args at odd sizes,wantsTransparentBackground, verification-page background), producer (chromaSubsamplinggates, capture surface, compile gate, render request, HTTP server), studio-server (encoder options, render route), CLI (plan, Docker argv, local render forwarding)--chroma-subsampling 444(yuv444p / High 4:4:4 Predictive, background unchanged, PNG capture logged), 4 workers, odd 1281x721 size, every rejection case; player tests abovecheck-comment-citationsandcomment-ratchetpassDeveloped on Windows: engine, producer, studio-server and CLI suites show no new failures against
main; the remaining failures there are Windows-only and also fail onmain(symlink permissions, POSIXsleep/echoin tests, path separators, ffmpeg 8+ dropping-vsync).🤖 Generated with Claude Code