[Feat] feature-demo skill: polished, narrated demo videos - #1177
[Feat] feature-demo skill: polished, narrated demo videos#1177mrubens wants to merge 36 commits into
Conversation
A new standard skill that produces Screen Studio-style feature demos: the demo script drives real browser interactions (delegated to the proof-runner subagent via agent-browser) while logging cursor, click, and element-rect timelines on the same clock, then a bundled Remotion composition post-produces the recording with zoom-to-element moves, click ripples, captions, and optional voice-over narration. - packages/cloud-agents .../feature-demo: SKILL.md orchestration, capture runner, narration + timing-fit scripts, render project (excluded from tsc/knip/oxlint like agent-browser; copied verbatim into the sandbox skills dir at activation) - apps/api /api/tts/narration: control-plane TTS endpoint (task-token policy + rate limit). ElevenLabs credentials live only on the control plane; R_ELEVENLABS_* are control-plane env names, stripped from sandbox env injection. Unconfigured deployments 404 and the skill degrades to captions-only. - apps/worker/Dockerfile: bake Remotion's chrome-headless-shell at /opt/remotion/headless-shell (the bundled Chrome removed old headless); runtime fallback via remotion browser ensure for older snapshots. - docs: cookbook recipe + environment variable reference.
|
No code issues found. See task
Reviewed 4597bcb |
| const js = `(function(){var e=document.querySelector(${JSON.stringify( | ||
| sel, | ||
| )});if(!e)return null;var r=e.getBoundingClientRect();return{x:r.x,y:r.y,w:r.width,h:r.height};})()`; |
There was a problem hiding this comment.
Addressed in b995257: the selector now travels into the page-eval as base64 with a strict alphabet check before interpolation, so no selector bytes can reach the constructed code.
eleven_v3 reads slowly and needs the 1.12 atempo speed-up; v2-family models pace naturally and should not be sped up. The narration endpoint now returns its modelId, build-narration records it in the manifest, and fit-timing derives the atempo default from it (argv still wins).
Per review direction: the bundled Remotion project is a starting point the agent copies and adapts (branding, captions, layout, presets), not a fixed pipeline stage. The timeline JSON stays the stable contract between capture and render. When adapting, the skill installs Remotion's official agent skills (remotion-dev/skills — their sanctioned successor to the deprecated Remotion MCP) for current API guidance. A render/ README documents the bug classes worth preserving (counter-scaled cursor, edge-clamp guard, timeline-driven interpolation).
Simplify per review: one model, one settings profile, no R_ELEVENLABS_MODEL knob and no model-aware atempo plumbing. multilingual_v2 reads voice clones naturally at a natural pace; the timing fit's atempo stays available as an explicit argv override.
- capture: bracket cursor motion with a hold key at motion start so the synthetic cursor no longer drifts toward its next target through waits, holds, and scrolls (and clicks on a fresh element get a short bracketed hop instead of a slow glide). - capture: pass selectors into the page-eval as alphabet-checked base64 so no selector bytes are interpolated into code (CodeQL). - fit-timing: solve the playback rate by simulating the schedule instead of a closed-form cumulative estimate; a line that waits for a late anchor ends later than the estimate, which could push later lines past their anchors. Lines intentionally START just before their zoom and play over its hold; the docs now say so explicitly.
The cookbook index is generated from recipe frontmatter; run the generator instead of hand-editing between the markers.
The binary name differs per platform: headless_shell on linux-arm64, chrome-headless-shell on linux64, so the hardcoded name failed the amd64 image build. Resolve it with a find over both names (validated under amd64 emulation and in the arm64 sandbox image).
Capture emits an explicit per-caption anchor (the zoom's landing time); the timing fit schedules against it. caption.start begins with the cursor glide ~moveMs earlier, so anchoring there started narration before the zoom even began moving. Also hoist the Dockerfile block comment above the RUN chain.
Field failure from a real game demo: CDP screencast only emits frames on visual damage and stamps them without wall-clock gaps, so a 52s interaction over a static-rendering surface collapsed into a sub-second video. The capture runner now injects an imperceptible 2px rAF ticker after record start so compositor damage flows at wall-clock rate (static-page regression: 7.93s timeline -> 8.0s recording), closes any pre-existing pages before recording (record start creates a fresh browser context; a page left over from inspection would split beats and recording across contexts), and fails loudly when the recording comes back much shorter than the interaction instead of letting a garbage video flow downstream. SKILL.md routes that failure to a 'stale sandbox runtime' blocker (old snapshots carry a 2018 legacy ffmpeg that crashes record stop under real frame volume).
Field-failure fix: static surfaces recorded as sub-second videosA real dogfood run (game demo) surfaced a capture failure mode none of the development surfaces hit: CDP screencast only emits frames on visual damage, so a statically-rendered app (games between state changes, idle dashboards) collapsed a 52-second interaction into a 0.8s video with no wall-clock mapping. Docs pages and fixtures masked this because scrolling and hover repaint constantly. Fixed in 8dc12b2, all reproduced/verified against live sandboxes:
|
Second dogfood run failed at the delegation seam in two ways: the brief's home-relative runner path did not resolve in the delegated runtime, and the proof-runner refused the capture script as a disallowed browser path (its instructions allow only the agent-browser CLI, and a per-brief assurance is not reliable authority). - The skill now stages the runner at /tmp/feature-demo/capture.mjs before delegating (no home-relative paths across the boundary) and parent-side paths use $HOME explicitly. - The proof-runner prompt carries a narrow standing exception: the staged runner is an agent-browser orchestrator and running it as the brief specifies is compliant; the exception names that one path and no other script.
Second dogfood failure: the delegation seamA second live run failed before capture in two ways, both at the parent→proof-runner boundary:
Fixed in b17b6c4 with prompt/skill test coverage for both. |
Live-sandbox controls showed the sparse-frames failure is page-specific: a WebGL game stalls headless frame production for the whole page while plain pages on the same sandbox record at full wall-clock rate. SKILL.md now names that blocker (webgl surface stalls headless recording) so agents diagnose by target instead of burning retries; the stale-runtime ffmpeg hint stays scoped to actual record-stop ffmpeg errors.
Review caught that the sanction trusted a parent-writable /tmp path: anything staged there would inherit the narrow agent-browser allowance. The worker now computes the sha256 of the activated feature-demo capture runner at config-generation time and pins it into the proof-runner prompt; the runner verifies the staged file's digest before executing and reports 'capture runner integrity mismatch' otherwise. Without a digest (skill absent) the sanction is omitted entirely.
Admins configure narration TTS under Settings → Integrations → ElevenLabs (API key + voice ID) instead of deployment env vars. Encrypted at rest, deployment-scoped, admin-only — same storage as Granola. New 'credential_only' serverMode: the integration exposes no MCP tools to agents and is explicitly excluded from getMcpServerConfigs, so the key is consumed only by the control-plane /api/tts endpoint and never travels toward a task sandbox (proven by an inverse delivery test). The TTS endpoint now resolves credentials from the connection first, falling back to R_ELEVENLABS_* env vars for self-hosters. UI, tRPC commands/procedures, hooks, icon, self-setup catalog metadata, and docs updated; env-var mocks in the TTS test replaced with a resolver mock.
The API server registers a proxy route for every non-native catalog entry, and route construction throws for an integration without an upstream URL — so adding credential-only ElevenLabs broke server construction (caught by the live-server integration suite). Add isCredentialOnlyMcpIntegration and skip those entries in the proxy registration loop, alongside native and Linear.
Completes the review point on bbcbb92: the worker's curated proxy-path map no longer carries a dead entry for integrations that are never delivered to sandboxes.
Review caught a TOCTOU in the digest sanction: sha256sum then node as separate commands over the parent-writable staging path leaves a swap window. The proof-runner now copies the staged file to a fresh private path, digests the copy, and executes exactly that verified copy — checked bytes are executed bytes, and the staging path is never re-read after verification.
Follow-up to 6b206ef: the first assertion still matched the old 'runner at' phrasing; the sanction now reads 'runner staged by the parent at'.
The prior mktemp copy still lived in shared /tmp, so a same-uid parent could swap it between sha256sum and node opening it — copying does not close a TOCTOU. The proof-runner now runs a single node process that reads capture.mjs exactly once, hashes those bytes, and executes the same bytes via a data: import only on an exact match. The file is opened once, so there is no post-hash window; the digest is still baked in at config-generation time. SKILL.md hands off paths and lets the proof-runner own the verified invocation.
A live dogfood run collapsed a 47s game interaction into a 0.7s clip: GPU-backed canvases (games, 3D, WebGL/WebGPU) never present to the headless compositor, starving the screencast. agent-browser's own docs prescribe `--headed` for exactly this, and on displayless Linux it auto-starts a private Xvfb (present in the worker image). The capture runner now prefixes --headed on every agent-browser command by default (HEADED=0 opts out); harmless on ordinary pages. Verified end-to-end in roomote-worker:arm64-real against an animated WebGL canvas: full pipeline, real-duration recording, auto-Xvfb, exit 0. SKILL.md updated; this supersedes the earlier 'file an upstream issue' plan — the flag already exists.
The generated wrapper stripped --headed and force-exported AGENT_BROWSER_HEADED=false, so the capture runner's --headed was a no-op (review catch). The wrapper now forwards --headed and exports AGENT_BROWSER_HEADED=true only when a caller explicitly passes it; default behavior is byte-identical (headless), preview-cookie seeding stays headless, and --headed false/=false explicitly opts out. Verified the arg handling in isolation across the command set.
… capture Forensics on a stalled demo capture (live worker container) traced the real root cause past the misleading agent narrative: worker containers run with no init process and a 512 pids-limit, so agent-browser daemon restarts leak Chrome process trees whose defunct children accumulate (observed 174 defunct, pids.current 494/512) until fresh Chrome dies with pthread_create: Resource temporarily unavailable — surfaced as Chrome exiting early and misread as a display problem. - Run docker workers and task daemons with --init (tini) so orphaned children are reaped. - Raise DOCKER_WORKER_PIDS_LIMIT default 512 -> 2048 (still overridable). - feature-demo capture: stop any existing agent-browser daemon (agent-browser close, not just close --all) before recording so --headed actually applies, and hard-fail when record start reports '--headed ignored: daemon already running' instead of silently recording a headless (stalling) session. Diagnosis from the Roomote-Evals forensic session; verified the code gaps directly. Capture path re-smoked end-to-end in the worker image including the daemon-already-running trap.
0.27 does not auto-start a virtual display for --headed, so headed recording died with 'Missing X server or $DISPLAY' (and, against an already-running daemon, '--headed ignored'). 0.33.2 auto-starts Xvfb on displayless Linux, which is what the feature-demo capture path needs. Verified in the worker image: headed recording of an ordinary page now succeeds end-to-end where 0.27 failed. (WebGL-heavy game surfaces can still hang the headed browser — a separate, known limitation the skill reports as a blocker.)
The demo's story beats, captions, and selector candidates are creative planning work, so the skill now delegates that step to the existing advisor subagent (R_PLANNING_MODEL when configured, else the coding model at the advisor reasoning level) with a complete brief. The parent keeps ownership of everything the advisor cannot do: verifying proposed selectors against real page source, writing demo-script.json, and the capture/render pipeline. Follows the harness advisor contract: internal guidance only, one retry on empty output, then plan directly.
Source-file selector verification is impossible for a public page the user names: no local source to grep, and the parent must not drive the browser. Split verification by surface — repository-backed surfaces grep source; external pages delegate a read-only resolve-only pre-flight to the proof-runner (open URL, snapshot/get box, confirm or correct each selector) before authoring the script.
Baking the chrome-headless-shell but not the render project's deps left every demo paying a multi-minute Remotion + React npm install. Bake node_modules at /opt/feature-demo/render from the skill's own pinned package.json (single source of truth, no drift), and have the render step reuse them (cp the baked node_modules into the work-dir copy); npm install is now only the fallback for an adapted composition that adds deps or an older snapshot. Validated in the worker image: rendered fully offline (npx --offline) reusing the baked modules, no install.
The previous pre-flight conflicted with the proof-runner contract (fixed browser target, one brief per task), so it was not actually usable. For an external public page the browser is reachable only through the single capture delegation, and the capture runner already resolves every selector live and fails loudly naming any that miss. Author best-effort resilient selectors and correct them from that named failure within the allowed retry, rather than inventing a second browser trip.
…tion Adapting an existing composition needs markup, render, and doc-lookup guidance — not the full remotion-dev/skills bundle (which also pulls create/maps/saas/upgrade/interactivity). Pin the install to --skill remotion-markup --skill remotion-render --skill remotion-docs --yes. Verified the exact CLI syntax in the worker image: installs exactly those three, non-interactively, project scope, no warnings.
Inspired by Mux's agent-video two-pass design, inverted to keep our post-production polish: narration is synthesized BEFORE capture, and the capture runner conducts the browser to it — each captioned beat holds for exactly as long as its line takes to speak (real clip duration when narrated; estimated speaking time from the caption text when the deployment has no TTS configured), the voice starts 0.4s before its zoom lands, and clip start times are stamped at the moment each zoom actually lands. Captions-only demos therefore read at narrative pace too, and fit-timing collapses from a reconciliation solver (atempo, playback-rate search, rescheduling) to a trim-only pass — that whole problem class is gone. Also settles the recording mode on evidence from repeated in-image runs: headless + frame ticker is deterministic at wall-clock rate (3/3 identical), while agent-browser 0.33.2 headed recording wedges its daemon on longer sessions regardless of ticker (3/3 failures, failure point moves). Default is now headless; '\''headed'\'': true in the demo script opts a GPU/WebGL surface into headed capture with an honest failure path. Validated end-to-end narrated in the worker image: clip starts land exactly at zoom-land minus the voice lead and the rendered mp4 carries the audio track at the stamped times.
| // headless compositor — opt in per demo with `"headed": true` in the script | ||
| // (or HEADED=1). Note: current agent-browser headed recording can wedge the | ||
| // daemon on longer sessions; use it only when the surface requires it. | ||
| const HEADED = script.headed === true || process.env.HEADED === '1'; |
There was a problem hiding this comment.
Setting "headed": true still cannot start a headed capture when preview cookie seeding is active. The wrapper treats record start as a seedable command and invokes the real CLI with AGENT_BROWSER_HEADED=false to set cookies in the default session before forwarding this command. That creates a headless daemon, so the following --headed record start is ignored and this runner aborts at the daemon-warning check. Seed cookies through the requested launch mode or isolate the seeding session so the advertised WebGL fallback can run.
There was a problem hiding this comment.
Confirmed — the seeding calls hardcoded AGENT_BROWSER_HEADED=false, so whenever seeding was what launched the daemon, a later --headed record start was silently headless (and my warning-string check then aborted, or worse, false-positived when the runner's own viewport call had already launched the daemon headed). Fixed in 3aa8cc9 on both layers: seed_preview_cookies now inherits the wrapped command's requested mode (AGENT_BROWSER_WANT_HEADED), and the runner replaced warning parsing with a positive mode check — after record start it reads navigator.userAgent and hard-fails a headed request only when the browser actually reports HeadlessChrome. Validated in the worker image with seeding active (ROOMOTE_WEB_HOST + bypass value set): headed opt-in records headed and passes the UA check; the headless default is unaffected.
…by UA Review caught that the wrapper's preview-cookie seeding always invoked the CLI with AGENT_BROWSER_HEADED=false before seedable commands, so when seeding was what launched the daemon, a later --headed record was silently headless. Seeding now inherits the wrapped command's requested mode. The capture runner also stops parsing the ambiguous 'daemon already running' warning (a false positive when the runner's own viewport call pre-launched the daemon headed) and instead verifies the browser's actual mode positively via its user agent — headless Chrome self-identifies as HeadlessChrome. Validated in the worker image with seeding active (preview host + bypass value set): headed opt-in records headed and passes the UA check; headless default unaffected.
Feedback from watching a produced demo: the constant zoom-to-element reads as artificial, and the story is too sparse. New beat grammar: 'show' (scroll the subject into view and narrate over it, camera wide) is the default narrated move; 'focus' zooms are reserved as the wind-up for an interaction the demo actually performs, so a zoom communicates 'watch this click'. The advisor now writes the narration first as a flowing 4-7 line spoken story (8-20 words per line, why-it-matters to payoff) instead of 3-9 word labels, and the speaking-time estimate clamp rises to fit. Validated in the worker image: a three-line demo runs 20.5s with exactly one zoom (the click wind-up), camera wide and scrolling everywhere else.
Narrative-length cuts (25-45s, 750+ frames, software rendering on a small container) routinely exceed the harness's default two-minute command window; agents were burning a first attempt on the timeout and retrying. Tell them to allocate ~10 minutes up front.
Two rulings from watching produced demos: on-screen captions must be the spoken lines verbatim (the lines.json divergence mechanism is removed), and narrated captions now highlight the word being spoken. The TTS endpoint uses ElevenLabs' with-timestamps variant and passes character alignment through; build-narration rolls it into per-word timings (with a sanity check that skips highlighting if alignment ever disagrees with the text); the renderer draws captions as word spans — active word glowing, spoken words bright, upcoming words dimmed. Word times are relative to clip starts, so audio-first stamping keeps the highlight accurate with no extra sync machinery. Captions-only demos render plain. Verified by rendering a timed line in the worker image and inspecting frames mid-line: the highlight sweeps with the audio.
Captions no longer overlay the recorded content: the canvas splits into a stage (where the window lays out and zooms) and a reserved caption band on the backdrop beneath it — the window is inset anyway, so the space was already paid for. Zoom translation clamps to the stage and a stage clip guarantees even a deep zoom never bleeds into the band. Deriving the stage-aware clamp also fixed a latent bug in the original edge-clamp bounds that could show backdrop gaps when zooming targets near the window edge. The demo script also gains a declarative captionStyle object — position top/bottom, active-word accent color, pill on/off (off = bare text with drop shadow), and a font-size multiplier — for brand-fit tweaks without touching the render template; deeper redesigns still go through the adaptation path. Verified by rendering wide and deep-edge-zoom frames in the worker image: band stays clean in both. Also: lint-staged oxfmt now passes --no-error-on-unmatched-pattern so a commit touching only formatter-ignored paths (like this skill's render template) is not rejected by the pre-commit hook.
|
|
||
| const SHOT_W = timeline.video.width; | ||
| const SHOT_H = timeline.video.height; | ||
| const usableH = captionBand > 0 ? stageH - pad : canvasH - 2 * pad; |
There was a problem hiding this comment.
The reserved band changes the wide preset's base window from 1344px to about 1205px. At the documented/default 1.5x focus scale, the window is then only about 1807px wide on a 1920px canvas, so the BASE_W * S >= canvasW clamp never runs and backdrop shows at the left/right during ordinary focus zooms. Previously the 2016px scaled window covered the frame. Preserve enough stage width (or ensure the zoom/clamp covers the stage) so the normal 1.5x focus does not regress into visible side gutters.
There was a problem hiding this comment.
Fixed in c02c34f. The band-shrunk window (1205px wide) at the default 1.5x scale covered only ~1807px, exposing side gutters — as you said. The fix floors the effective scale at full-width coverage (canvasW / BASE_W) whenever a zoom passes the vertical-cropping threshold (stageH / BASE_H), ramping in smoothly over 0.15 scale units, so the view is always either a floating uncropped window or a full-bleed zoom, never a cropped hybrid. Show-beat glides drop to 1.12x to stay below the cropping threshold and keep floating.
One subtlety the fix surfaced: the floor lands exactly on canvasW / BASE_W, and in floating point BASE_W * S evaluates one ulp short of canvasW (1205 * (1920/1205) = 1919.9999999999998), which silently skipped the edge-clamp gate and let the raw center-pull expose a gutter anyway. The floor now overshoots by 0.002 and the clamp gates get a half-pixel epsilon.
Validated in the worker image with the reviewer scenario (default 1.5x focus + a show glide, captions-on): frames early/mid/late in the zoom hold are all full-bleed edge to edge, and the glide floats uncropped.
There was a problem hiding this comment.
The ramp still leaves a cropped-but-not-full-width interval during every normal focus animation. In the wide captioned preset, sRaw = 1.2 yields S ≈ 1.307: the window is 984px tall (past the 873px stage, so it is clipped) but only 1575px wide (short of the 1920px canvas). The side gutters therefore remain visible while the zoom crosses this range, despite the comment claiming the view is never a cropped hybrid. Snap to sCover once sRaw reaches sCrop, or delay clipping until horizontal coverage is reached.
There was a problem hiding this comment.
You were right that the ramp still animated through the bad state, and chasing it turned up why the floor was the wrong shape of fix entirely.
In the wide preset the window starts cropping vertically at 1.16x (with a caption band reserved; 1.29x without one) but does not reach the canvas edges until 1.59x. Every scale in between is cropped AND narrower than the canvas, so a zoom cannot get from rest to full-bleed without crossing it — no ramp avoids that. Full-bleed is not reachable cleanly in that geometry at all, and this predates the caption band; the band only widened the gap enough to put the resting default 1.5x inside it.
So 4597bcb caps the zoom at the largest scale that keeps the whole window on the stage, less an 8px inset so a sliver of backdrop always remains. The window is then always a complete rounded card, never a crop with backdrop down one side. Presets where coverage comes first are untouched: vertical covers the width at 1.20x and does not crop until 2.58x, so it still goes full-bleed. The tradeoff is that a focus zoom is now a gentle push-in rather than a punch to full screen, which suits a skill that already treats zooms as rare wind-ups; glides drop to 1.06 so a focus beat still reads as distinct, and SKILL.md documents the cap.
The edge clamps also lost their gates. The same pair of bounds means 'keep covering the stage' when the scaled window is bigger than it and 'stay inside the stage' when it is smaller — they just swap order — so clamping unconditionally is right in both regimes, and it fixes a latent case where a nearly-stage-height window could drift into the caption band on the center pull. The floating-point epsilon from the previous commit is gone with the floor.
Verified over every frame rather than samples: a render exercising both documented zoom extremes (1.5 and 1.9) is probed at the stage's left and top edge strips and each frame classified. 339/339 frames floating, 0 hybrid. As a control, the same probe finds 143 hybrid frames in the previous build, beginning at frame 43 — right inside the ease, which is exactly what you flagged.
The reserved caption band shrinks the window enough that a default 1.5x focus zoom cropped vertically without covering the canvas width, exposing backdrop side gutters. Floor the effective scale at full-width coverage once a zoom passes the cropping threshold, so the view is always either a floating uncropped window or a full-bleed zoom. The floor lands exactly on canvasW / BASE_W, where floating point can leave BASE_W * S one ulp short of the clamp gate — overshoot the floor slightly and give the gates a half-pixel epsilon. Show-beat glides drop to 1.12x to stay below the cropping threshold. The window's drop shadow moves outside the stage clip so it falls on the backdrop instead of shearing at the stage edge.
Replaces the navy gradient with the brand color from the web app theme (--background-new, ~#c9f24d) with a subtle radial falloff.
The coverage floor fixed the resting frame but not the animation: in the wide preset the window starts cropping vertically at 1.16x (with a caption band; 1.29x without) and does not reach the canvas edges until 1.59x, so every scale in between is cropped AND narrower than the canvas, showing backdrop down one side. A zoom cannot get from rest to full-bleed without crossing that range, so no ramp shape avoids it — full-bleed is simply not reachable cleanly in that geometry. Cap the zoom at the largest scale that keeps the whole window on the stage, less an 8px inset so a sliver of backdrop always remains. Presets where coverage comes first (vertical covers the width at 1.20x and does not crop until 2.58x) are unaffected and still go full-bleed. The edge clamps drop their gates: the same pair of bounds means keep covering when the window exceeds the stage and stay inside it when it does not, so clamping unconditionally is correct in both regimes and also stops a nearly-stage-height window from drifting into the caption band. Glides drop to 1.06 so a focus beat still reads as a distinct push-in. Verified in the worker image over every frame of a render exercising both documented zoom extremes (1.5 and 1.9): 339/339 frames floating, 0 hybrid. The same probe finds 143 hybrid frames in the previous build.
What
A new
feature-demostandard skill that produces polished, Screen Studio-style demo videos of product features: a real recording driven live in the sandbox browser, post-produced with smooth zoom-to-element moves, a synthetic cursor, click ripples, captions, and optional voice-over narration.Invoke with
$feature-demoor by asking for a demo video in a task.How it works
One declarative demo script drives everything. The capture runner performs the real browser interactions (scroll, hover, click, type) via the
agent-browserCLI and records, on the same clock, where the cursor is, when clicks land, and the resolved rectangle of every focused element. The bundled Remotion composition consumes that timeline, so a zoom can never drift from the element it targets.Pipeline: author script → capture (delegated to the
proof-runnersubagent, keeping browser containment intact) → narrate (optional) → fit timing → render → verify → upload viamanage_artifacts.The bundled Remotion project is a reference template, not a fixed pipeline stage: the agent copies it into the work dir and adapts the copy (branding, caption styling, layout, presets) when a demo calls for it, installing Remotion's official agent skills (
remotion-dev/skills— their sanctioned successor to the deprecated Remotion MCP) for API guidance. The timeline JSON is the stable contract between capture and render, and arender/README.mddocuments the hard-won invariants worth preserving (counter-scaled cursor, edge-clamp guard, timeline-driven interpolation).The timing fit encodes what makes these demos feel produced rather than recorded: the dead opening hold is trimmed, the voice-over is paced into a conversational band (pitch-preserving
atempo), and the recording's playback rate is solved so each spoken line finishes just before its zoom lands.Narration without the key in the sandbox
Narration uses ElevenLabs, but the provider key never enters a task sandbox:
POST /api/tts/narration(route policytask-token, rate-limited, streamed body cap). The sandbox posts plain caption text with its run-scoped token; the control plane holdsR_ELEVENLABS_API_KEY/R_ELEVENLABS_VOICE_IDand returns audio. Same posture as the inference gateway and custom MCP proxy.R_ELEVENLABS_*are registered as control-plane env names, so they are reserved from the generic env editor and stripped from sandbox env injection.Worker image
Remotion cannot render with the bundled agent-browser Chrome (Chrome 143 removed old-headless), so the image now bakes Remotion's
chrome-headless-shellat/opt/remotion/headless-shell(both architectures, pinned, arch-independent symlink). Older snapshots fall back tonpx remotion browser ensureat render time — the smoke run below exercised exactly that fallback.Testing
pnpm lint,pnpm check-types,pnpm knipgreen across workspaces.npm install+ headless-shell fallback → rendered a clean 7.6s 1080p mp4, verified with ffprobe and frame inspection.Docs
environment-variablesreference for the two new optional vars, including the recommendation to scope the ElevenLabs key to text-to-speech only.