Skip to content

fix(cli): long projects check to completion: boxed resumable copies, --prepare-media, idle deadline - #6104

Open
miguel-heygen wants to merge 12 commits into
mainfrom
fix/check-copies-at-composition-size
Open

miguel-heygen wants to merge 12 commits into
mainfrom
fix/check-copies-at-composition-size

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

What

hyperframes check now makes the browser-playable copy of an HEVC/ProRes video at the size that fills the composition, instead of at the source size. A 4K clip in a 1920x1080 project is copied at 2573x1448 (the composition's box on the preview size ladder) rather than 3840x2160. Copies for preview and publish are unchanged. It also clears partial copies that a stopped process left behind, once they have been untouched for 10 minutes (the old limit was an hour).

Two more changes cover a clip too long to copy inside a caller's time limit: a source over 90 seconds is copied in one-minute pieces that stay in the cache, so a stopped copy continues from its finished pieces instead of starting over; and hyperframes check --prepare-media makes only the copies and exits (no browser, no report), so a caller can run that step first without the check's time limit.

Why

A check on a long 4K60 HEVC project spent its time in "making browser-playable copies". The copy re-encodes every frame at the source size, and a stopped run (a runner time limit, a closed terminal) throws the partial copy away, so the next run starts again. The page the check opens asks for a copy that fills the composition, so the 4K pixels were never needed there.

On the devbox (8 cores) a 4K60 HEVC copy runs at about 0.6x realtime, so 400 s of footage takes about 11 minutes at the source size.

How

  • The check server (serveStaticProjectHtml) now reads hf-proxy-box the way Studio's preview route does and asks resolveProxy for that copy. Before, it always asked for the source-size copy, so the page's request would have missed the boxed copy made up front. An invalid box, or a box without hf-proxy, answers 404 as in the preview route. This is the one place the page's request and the up-front copy now share a cache key. The seven callers of the server (check, snapshot, validate, compare and the others) now get boxed copies for the box their page asks for, the same copies Studio makes.
  • preResolveHostileMediaProxies finds the composition's size (findCompositionDimensions), rounds it up the existing preview size ladder (quantizePreviewProxyBox, now exported from @hyperframes/core) and asks for that box. The cache-hit check uses the same box, so a repeat check finds the copy.
  • resolveProxies takes an optional box per source and passes it to resolveProxy, which already names and sizes a boxed copy.
  • resolveProxies first sweeps its cache folder for dead partial copies (.tmp-* untouched for 10 minutes, or less if HYPERFRAMES_PROXY_CACHE_STALE_TEMP_MS is set lower). A copy being written keeps growing its file, so a live one is kept; a copy whose ffmpeg is suspended for 10 minutes (a laptop asleep) loses its partial and is made again.
  • A project with no composition size, or a video larger than the stage on the page, behaves as before.
  • Server pre-warm. serveStaticProjectHtml pre-warms copies when it opens; it now asks for the page's box (findStageProxyBox, the same helper the up-front copy uses) instead of the source-size copy, so a check no longer starts a full-size encode next to Chrome. The other six callers of the server get the same.
  • Idle deadline. The check no longer ends 120 s after Chrome was ready whatever it is doing. CheckDeadline is an idle clock: every phase entered, every sample finished, every operation that settles and every overlap batch restarts the 120 s window, and the check ends only after 120 s with none. The overlap pass was one page call over every time (up to ~240) with no way to report progress, which is where a ~70 s 1080x1920 project with 8 sub-compositions ran out at ~104 s while three --range slices passed; it now asks the driver for batches of eight and reports between them. A hung page still ends 120 s after its last progress, and the message reads "no progress for 120 s after Chrome was ready".
  • Resumable pieces. A source longer than 90 seconds is encoded in one-minute pieces (-ss/-t), each written under a temp name and renamed to a stable .part-<key>-<n> name when finished, then joined with the concat demuxer without re-encoding. A copy that is killed, aborted or out of its 15 minute budget loses at most the piece in flight, and the next run (check, preview or publish, since they share the one transcoder) continues from the first missing piece. Shorter sources keep the single pass. Finished pieces are swept after an hour untouched and never while their copy is running (the 10 minute limit above applies to temp files only, otherwise it would delete the first pieces of a long copy still encoding).
  • check --prepare-media [--json]. Makes only the copies, bounded per file by the existing 15 minute limit, then exits: no browser, no report, same project lock as a check, honors --proxy/--no-proxy. --json prints { ok, prepared: [{ file, state: "ready" | "cached" | "failed", reason? }], total, ready, failed }; progress stays on stderr. Exit 0 when every copy is ready, 1 when any failed. Documented in docs/packages/cli.mdx.

Evidence

Synthetic 30 s 4K60 HEVC clip with audio in a 1920x1080 project; hyperframes check built from main and from this branch on the same 8-core machine (the branch run was on a busier machine):

main this branch
copy made before the page opens 49.8 s 32.0 s
size of the copy 145 MB 39 MB
files in the cache after the check 1 1
requests the page made for its copy 23 18, answered from the boxed copy
whole check 58 s 42 s
second check (copy reused) not measured 8 s

About 1.5 to 1.9x less time for a 1080p composition (the ladder rounds 1080p up to 2573x1448, not 1080p); a 4K composition gets no change. For 400 s of 4K60 that is roughly 6 minutes instead of 11 on this machine, so a slower machine can still pass a runner's limit. Resumable pieces and --prepare-media are covered below.

Resumable pieces, synthetic 200 s 1080p HEVC with audio, Linux, ffmpeg 7.0:

Run Result
Uninterrupted, 4 pieces 50.4 s; 6000 video frames, container 200.02 s
Process and ffmpeg killed at 22 s, then rerun rerun 29.1 s (22 + 29 is about 51: no restart); 6000 frames, 200.02 s, same size as the uninterrupted copy
Real check --prepare-media --json, stopped at 25 s, rerun rerun 35 s, state: ready, exit 0; a third run reports cached in 3 ms

Test plan

  • Unit tests added/updated

  • Manual testing performed

  • Documentation updated (if applicable)

  • Comments follow CONTRIBUTING.md "Comments": they say why, not what, and a bug fix says what the code must do and how to reproduce the bug

  • checkBrowser.test.ts: the copy is requested at the composition's box (2048x1448 for 1920x1080) and the cache check uses the same box. Fails without the fix.

  • proxyTranscoder.test.ts: resolveProxies makes a batch's copy at the box it is given, under that box's own cache name; and it removes an 11-minute-old partial while keeping a 1-minute-old one. Both fail without the fix.

  • staticProjectServer.test.ts: a page request with hf-proxy-box=2048x1448 resolves that box; an invalid box and a box without hf-proxy answer 404. Fail without the change.

  • Each of the five failed without the change and passes with it; studio-server (53 tests), checkBrowser and staticProjectServer (82) passed 3 runs in a row. tsc for cli, studio-server and core, oxlint, oxfmt, fallow and the commit hooks pass.

  • Pieces: the rerun encodes only the missing pieces, then joins and leaves only the final file; a short source stays one pass; stale pieces are swept after an hour but not while their copy runs; --prepare-media enters through createCheckCommand (no check pipeline call, JSON shape, exit 1 on a failed copy). Removing the "skip a finished piece" line turns the resume test red (expected [ '0', '60', '0', '60', '120' ] to deeply equal [ '0', '60', '60', '120' ]); removing the in-flight guard turns the sweep test red. cli check tests (170) and studio-server helpers (523) passed 3 runs in a row; typecheck, oxlint, oxfmt and fallow pass.

Pre-warm: mediaProxyPreview.test.ts (the real pre-warm passes the box to resolveProxy) and staticProjectServer.test.ts (opening the server passes the composition's box to the injection); putting the unboxed call back turns both red. Idle deadline: new tests in checkDeadline.test.ts (keeps going while steps make progress then ends; a settled operation restarts the clock; the message names the phase) and check.test.ts (a 10-minute-class overlap pass and slow samples that each finish complete without check_runtime_failure). Each fails when its piece is removed: enter() no longer restarting the clock (8 red), overlap grid in one call (1 red), no per-sample progress (1 red). 136 tests green x3 including the real-Chrome process tests (checkDeadline.process.test.ts), tsc clean. Not exercised: a 4K60 source through the pieces, audio by ear at piece joins (frame count and duration only; the audio ends 21 ms long over 3 joins), a caller that SIGKILLs only the CLI (its ffmpeg runs to the end of the current piece and its temp file is swept after the limit), a 4K composition, a project whose videos are larger than the stage, and any Windows run.

Size budget

@hyperframes/studio-server packed budget 141718 -> 142100 bytes (+382). The resumable 60 s copy pieces (proxyTranscoder.ts, proxyCache.ts) are real shipped code; packed size was 142043 (+325 over).

@vanceingalls vanceingalls left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at fdd986f7.

Requesting changes for one blocker: the check page never fetches the boxed copy this PR makes up front.

What works: the producer side is fine. resolveProxies passes box through to resolveProxy (proxyTranscoder.ts:681), and proxyCached checks the same box, so a repeat check skips the up-front copy. Making the copy at the composition's box is the right idea.

Blocker: the check server drops hf-proxy-box, so the page's request misses the boxed copy

preResolveHostileMediaProxies now makes the copy under the boxed cache name (checkBrowser.ts:170-178). But the server the check page talks to is serveStaticProjectHtml, and its proxy handler always resolves the copy without a box:

// packages/cli/src/utils/staticProjectServer.ts:239
const proxyPath = await resolveProxy(projectDir, filePath, variant, undefined, { signal });

The check page is a top-level page, so the runtime sends the box: displayScale = 1 at mediaProxy.ts:68, shownBox at mediaProxy.ts:327-330. The request is clip.mp4?hf-proxy=h264&hf-proxy-box=2048x1448, and the server reads only hf-proxy. Studio's preview route reads the box (routes/preview.ts:652); this server never did.

I confirmed it with a throwaway test in staticProjectServer.test.ts, using the file's existing mocks:

  • Request: fetch(\${server.url}clip.mp4?hf-proxy=h264&hf-proxy-box=2048x1448`)`.
  • Expected: resolveProxy(projectDir, clip.mp4, "h264", { width: 2048, height: 1448 }, …).
  • Received: the box argument is undefined.

What that means:

  • On main: the up-front copy and the page's request both use the source-size name, so the page gets a cache hit.
  • On this branch: the up-front copy is boxed, but the page's request goes to the source-size name. It misses and starts the full-size 4K transcode, the work this PR is meant to avoid.
  • Effect on the check: either the check waits for both copies (slower than main), or it samples videos the browser can't decode. The second case would happen if that transcode is cancelled when the server closes, which would also explain "1 copy in the cache after the check" in your table. That table measures only the up-front step, not the page's request.

Suggested fix:

  • Parse the box in serveProxyRequest the same way preview.ts does: parsePreviewProxyBox(params.get(PREVIEW_PROXY_BOX_PARAM)). Pass it to resolveProxy.
  • Add a test that links the two halves. For example, after preResolveHostileMediaProxies, the page's ?hf-proxy=…&hf-proxy-box=… request should resolve to the same getProxyCachePath(…, box). Today's tests mock each side separately, which is why this passes.
  • This changes all seven callers of serveStaticProjectHtml, so it's worth a look that snapshot, validate and compare are fine getting boxed copies.

Nits

  • proxyTranscoder.ts:668-673: passing DEAD_TEMP_MS explicitly overrides the HYPERFRAMES_PROXY_CACHE_STALE_TEMP_MS env knob on this path (proxyCache.ts:51, :153). If the knob is meant to apply here, use min(env, 10m) or leave it unset.
  • The 10-minute "untouched means dead" rule can delete a live temp file. Example: Studio's ffmpeg is suspended while a laptop sleeps, and a check then runs on the same project. ffmpeg's rename then fails, and the next request makes the copy again. That's rare, and the fallback is acceptable, but it's worth a comment.

Verdict: REQUEST CHANGES
Reasoning: The check server ignores hf-proxy-box, so the page's request misses the boxed copy made up front and starts the full source-size transcode. Shown by a unit probe against serveStaticProjectHtml.

— Via

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 271 (base branch 271), smooth 150 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

@vanceingalls vanceingalls left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at c78aa218 (delta since fdd986f7).

Fixed: the first blocker. serveStaticProjectHtml now reads hf-proxy-box and passes it to resolveProxy (staticProjectServer.ts:286-292, :245). It returns 404 for a box that isn't a rung, or a box without hf-proxy, matching Studio. The new tests pin both. The cleanup threshold now respects the env knob, using min(env, 10m). Thanks.

Blocker: the server's own pre-warm still starts the source-size transcode on every check

serveStaticProjectHtml still pre-warms without a box:

  • staticProjectServer.ts:272 calls injectMediaCodecMap(html, projectDir, [{ html }], undefined, /* prewarm */ true, copies.signal).
  • That function (studio-server/src/helpers/mediaProxyPreview.ts:111-118) calls resolveProxy(projectDir, src, proxyVariantFor(facts), undefined, { signal }) for every asset where shouldPrewarmProxy. That includes HEVC and ProRes (mediaCodecMap.ts:59-60).

On main, the check makes the source-size copy up front, so this pre-warm finds it in the cache and does nothing.

On this branch, the check only makes the boxed copy up front. So when runBrowserCheck opens the server (checkBrowser.ts:258), the pre-warm misses the cache (proxyTranscoder.ts resolveProxy: existsSync(cachePath) is false) and starts a full source-size 4K transcode in the background. It runs alongside Chrome for the whole check. copies.abort() stops it when the server closes (staticProjectServer.ts:351/363), so it never finishes or gets cached, and it starts again on every check. Main's second check runs no transcode at all.

That brings back the cost this PR removes: a full-resolution 4K HEVC encode competing for the runner's CPU during sampling, which is the runner time-limit case in the PR body. The "second check (copy reused) 9 s" row was measured with this background encode running and then killed.

The same mismatch now reaches the other six callers of serveStaticProjectHtml (snapshot, validate, compare, …). Their pages now get the boxed copy, but their pre-warm still makes the source-size one, which nothing will ask for.

Suggested fix: pre-warm the copy the page will ask for. Pass the stage box (quantizePreviewProxyBox of findCompositionDimensions(html)) through injectMediaCodecMapIntoHtml to its resolveProxy call. Alternatively, let runBrowserCheck turn the pre-warm off, since it has already made the copies.

Please add a test. For example: after preResolveHostileMediaProxies has made the boxed copy, opening serveStaticProjectHtml should not call resolveProxy with an undefined box.

CI

Tests on windows-latest: engine-cli is red at this head. The @hyperframes/cli test process Exited with code 1, but the job log shows no failing test and no summary, so I couldn't tell whether this change caused it. Worth checking the test-step-log-engine-cli artifact, or re-running.

Verdict: REQUEST CHANGES
Reasoning: The check page now gets the boxed copy, but the check server's pre-warm still starts the source-size transcode in the background on every check. That repeats the cost the PR removes, and on repeat checks it is new work main doesn't do.

— Via

…stopped copy resumes

A long source is copied in one-minute pieces that stay in the cache, so a copy
that is killed or runs out of time continues from its finished pieces.
check --prepare-media runs only that step: no browser, no report, a per-file
result as JSON, and exit 1 when a copy failed.
The short limit for dead partial copies would delete the first pieces of a
long copy that is still encoding. Finished pieces now follow their own
one-hour limit and are never swept while their copy is in flight.
@mintlify

mintlify Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
hyperframes 🟢 Ready View Preview Oct 23, 2026, 1:48 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

@miguel-heygen miguel-heygen changed the title fix(cli): check copies hostile videos at the composition size, not the source size fix(cli): long hostile videos check in time: smaller, resumable copies and check --prepare-media Oct 23, 2026
The comment gate limits a comment line to 120 characters and a run to four lines.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES · 1 blocker · 1 important · 0 nits — reviewed at 90557abe.

Blocker, still open from @vanceingalls's review at c78aa218: the check server still pre-warms the source-size copy. staticProjectServer.ts:272 → injectMediaCodecMapIntoHtml (mediaProxyPreview.ts:113-118) calls resolveProxy(..., undefined /* box */, ...) for every HEVC/ProRes file, so every check starts a full 4K encode next to Chrome. The resumable pieces make this worse. Each check's aborted pre-warm now leaves finished source-size .part-* pieces, so repeat checks keep moving that unwanted encode forward. Those pieces stay out of the size cap until they've gone an hour untouched (readCacheInventory skips live leftovers). Fix: pre-warm at the stage box, or let runBrowserCheck/--prepare-media callers turn the pre-warm off. Add a test that opening the server after preResolveHostileMediaProxies makes no resolveProxy call without a box.

Important (CI): the required Comments check fails at this head. comment-ratchet reports that the comment share went up in mediaMetadata.ts (+1 line) and proxyCache.ts (+3 lines).

Scope

Read the whole diff: check.ts --prepare-media, checkBrowser box/files, staticProjectServer box parsing, proxyCache piece sweep, proxyTranscoder chunking/concat. The box fix from c78aa218 holds.

Not blocking, worth a thought: two processes (Studio + check) chunking the same cache key can race. Whichever finishes first unlinks the pieces (encodeInChunks) while the other is still concatenating them. On Linux the second copy fails once and the next request hits the cache. On Windows unlinkSync of a piece that is still open throws EBUSY and drops the finished copy. Other CI runs were still in progress when I reviewed.

The comment ratchet lets a file's comment share only go down.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES · 1 blocker · 0 important · 0 nits — 1 resolved, 1 open, 0 new · 90557abe → 415bf3db

✅ The Comments check passes now (the commit only removes comments). The blocker from my last review is still open: staticProjectServer.ts:272 still calls injectMediaCodecMap(..., true /* prewarm */, ...), and mediaProxyPreview.ts:114-118 still calls resolveProxy with no box.

…the source-size one

The server started a source-size copy of every hostile video next to Chrome on
each check, because its pre-warm passed no box while the page and the up-front
copy use the composition's box. The pre-warm now takes the same box, found by
the one helper the up-front copy uses.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE · 0 blocker · 0 important · 0 nits — 1 resolved, 0 open, 0 new · 415bf3db → 72ae3495

✅ Pre-warm blocker fixed. staticProjectServer.ts:280 now passes findStageProxyBox(html), and mediaProxyPreview.ts:119 forwards it to resolveProxy. The check server warms the same boxed copy that preResolveHostileMediaProxies made, so it no longer starts a source-size encode. Both new tests pin the box, and CI is green at this head.

The check ended 120 s after Chrome was ready no matter what it was doing, so a
long project whose sampling and overlap pass each kept working still failed
(the overlap pass was one page call that reported nothing for its whole run).

The deadline is now an idle clock: every phase entered, every sample finished,
every overlap batch of eight times and every operation that settles restarts it,
and the check ends only after 120 s with none. The overlap grid is asked for in
batches so it can report between them.
@miguel-heygen miguel-heygen changed the title fix(cli): long hostile videos check in time: smaller, resumable copies and check --prepare-media fix(cli): long projects check to completion: boxed resumable copies, --prepare-media, idle deadline Oct 23, 2026
The comment gate allows 120 characters on a comment line.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES · 1 blocker · 0 important · 0 nits — 0 resolved, 0 open, 1 new · 72ae3495 → b501527e

The idle deadline works. Progress only comes from enter() and the two coarse deadline.run() calls, so a hung page or a hung collectSampleAt still expires after 120 s, and rearm is a no-op before start and after close. But required Test (runtime) fails at this head (↓ inline).

}

/** Reruns content_overlap on a fine grid, unconditionally rather than gated on fingerprint change, since an animation aliased to the sparse grid collides between identical-fingerprint samples. */
async function collectOverlapGridInBatches(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blocker: batching the dense overlap grid in groups of 8 breaks checkBrowser.test.ts:344 ("runs the dense overlap grid in one page task…"). It now sees 10 page tasks instead of 1 (expected 10 to be 1), and that fails required Test (runtime). That test pins one page task for the whole grid. Either update it to the new contract (one task per OVERLAP_BATCH, sample times kept on what each task logs) or keep one task and report progress another way.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES · 2 blockers · 0 important · 0 nits — 1 resolved, 0 open, 2 new · b501527e → 611175a0

✅ The runtime test now states the batched contract: at most 8 per task, ceil(n/8) tasks, and each queued message carries its own batch's range. Test (runtime) passes. Two required checks fail at this head:

  • blocker (Typecheck): checkBrowser.test.ts:344. The cli lib has no Object.groupBy, so tsc reports TS2550, then TS7006 for calls and TS2339 for .length at 344–345. Count per task with a Map or reduce instead.
  • blocker (Build): verify:packed-manifests reports @hyperframes/studio-server packed at 142043 bytes, over its 141718 budget. Build passed at b501527e and this commit only touches a test, so the merge with the current main is what crosses the line. Rebase, then trim, or raise the budget in scripts/package-size-budgets.json with the reason in the PR body.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE · 0 blocker · 0 important · 0 nits — 2 resolved, 0 open, 0 new · 611175a0 → fe4b2e54

✅ Typecheck: the test counts per task with a Map, with no Object.groupBy. ✅ Build: the @hyperframes/studio-server packed budget goes from 141718 to 142100 (+382 B), with the reason in the commit and body. With main merged, every non-test source file carries the same patch as at 611175a0. Lint, Typecheck, Build and every Test job passed at this head.

@miguel-heygen
miguel-heygen dismissed stale reviews from vanceingalls and vanceingalls October 23, 2026 14:45

Addressed: the pre-warm blocker was fixed in a later commit and re-reviewed at the current head.

This branch was successfully deployed

1 active deployment
staging - docs — fe4b2e54 Deployed Oct 23, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants