Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion electron/ai-edition/deep-agent/service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,7 @@ export const TOOL_DESCRIPTIONS: Record<string, string> = {
setAspectRatio:
"Set the shape of the exported video: 16:9 for X, YouTube or a slide, 9:16 for a vertical story or short, 1:1 or 4:5 for a square feed, auto to follow the recording, or one of the clips' own shapes (getCurrentDocument's settingsContext.originalAspectRatios). Annotations on the frame keep their place and shape. The same as setEditorSettings with aspectRatio alone.",
setEditorSettings:
'Change the project\'s look: pass only the settings to change, named as getCurrentDocument returns them under `settings`. Background: wallpaper (a bundled path from settingsContext.bundledWallpapers, a CSS colour or gradient, or a base64 PNG/JPEG data URL; never a file path or web URL; an embedded image reads back as a marker, leave the key out to keep it), wallpaperMotion (moves a gradient or image only), backgroundBlur 0–1. Frame: aspectRatio ("auto", a preset, or an originalAspectRatios token; a fixed format picked for the first time fills the frame with a window that follows the cursor, which formatFollowCursor turns off when settingsContext.formatFill is "available"), padding 0–100 %, borderRadius 0–64 px, shadowIntensity 0–1, frame (none, window, laptop, phone, monitor) and frameTheme (light, dark), motionBlurAmount 0–1, depthOfField (blurs 3D-tilted zooms only). Camera: webcamLayoutPreset, webcamMirrored, webcamReactiveZoom (shrinks on zoom), webcamMaskShape (square or rectangle, the camera\'s own proportions; an untouched roundness follows it), webcamRoundness 0–1, webcamSizePreset 15–60, webcamAnchor (a corner or an edge middle), webcamCropRegion (the framing: a square share of the picture, width = height ≥ 1/3) and webcamCropPan (where that window sits, 0–1 per axis, 0.5 centred), webcamBackgroundMode (none, transparent, blur, custom: needs settingsContext.cameraSegmentation ready), webcamBlurIntensity 0–1, webcamWallpaper (as wallpaper). Cursor, under `cursor`: show, autoHide, theme (settingsContext.cursorThemes), asArrow (the recorded cursor kinds drawn as the arrow instead), size 1.5–6, smoothing 0–1, motionBlur 0–1, clickBounce 0–2, clickImpact, clickSound (only heard when the take recorded clicks) and clickSoundGainDb −24–0, model3d. Also autoFocusAll (every zoom follows the cursor) and audioGainDb −12–12 dB, the master volume. A value out of range or a key not listed is refused with the reason, and nothing changes. The result lists what was applied, including what the editor changes with it, and `inactive` for any of those the current state leaves without effect.',
'Change the project\'s look: pass only the settings to change, named as getCurrentDocument returns them under `settings`. Background: wallpaper (a bundled path from settingsContext.bundledWallpapers, a CSS colour or gradient, or a base64 PNG/JPEG data URL; never a file path or web URL; an embedded image reads back as a marker, leave the key out to keep it), wallpaperMotion (moves a gradient or image only), backgroundBlur 0–1. Frame: aspectRatio ("auto", a preset, or an originalAspectRatios token; a fixed format picked for the first time fills the frame with a window that follows the cursor, which formatFollowCursor turns off when settingsContext.formatFill is "available"), padding 0–100 %, borderRadius 0–64 px, shadowIntensity 0–1, frame (none, window, laptop, phone, monitor) and frameTheme (light, dark), motionBlurAmount 0–1, depthOfField (blurs 3D-tilted zooms only). Camera: webcamLayoutPreset, webcamMirrored, webcamReactiveZoom (shrinks on zoom), webcamMaskShape (square or rectangle, the camera\'s own proportions; an untouched roundness follows it), webcamRoundness 0–1, webcamSizePreset 15–60, webcamAnchor (a corner or an edge middle), webcamCropRegion (the framing: a square share of the picture, width = height ≥ 1/3) and webcamCropPan (where that window sits, 0–1 per axis, 0.5 centred), webcamBackgroundMode (none, transparent, blur, custom: needs settingsContext.cameraSegmentation ready), webcamBlurIntensity 0–1, webcamWallpaper (as wallpaper). Cursor, under `cursor`: show, autoHide, theme (settingsContext.cursorThemes), asArrow (the recorded cursor kinds drawn as the arrow instead), size 1.5–6, smoothing 0–1, motionBlur 0–1, clickBounce 0–2, clickImpact, clickSound (only heard when the take recorded clicks) and clickSoundGainDb −24–0, model3d. Also autoFocusAll (every zoom follows the cursor), audioGainDb −12–12 dB, the master volume, and recordingMuted (silences the recording\'s own sound in the preview and the export; added audio tracks still play). A value out of range or a key not listed is refused with the reason, and nothing changes. The result lists what was applied, including what the editor changes with it, and `inactive` for any of those the current state leaves without effect.',
addEffect:
"Write a visual effect as a shader and pose it over a span of the edited timeline (virtual seconds), or pose one of this project's effects again (effectId). Use it for a look the settings do not offer. `effect.code` is WGSL that defines `fn effect(uv: vec2<f32>) -> vec4<f32>`, called for every pixel; uv is 0..1 with (0,0) top-left. Available inside it: `frame(uv)` samples the composed picture; `footage(p)` samples the recording itself at its own resolution at point p of its surface (the space of `fx_to_footage`), transparent outside 0..1, with no cursor or webcam on it: read it for anything that magnifies, frame() for the rest; `params.<name>` reads each declared parameter; `ctx.time` is programme seconds; `ctx.region.x` goes 0 to 1 across the region (y its length in s, z seconds since it began); `ctx.resolution` and `ctx.aspect`; `ctx.cursor` (xy in uv, z = 1 when on screen); `ctx.clicks[0..3]`, the last four clicks, newest first (xy in uv where it happened, z seconds since, w = 1 when the slot holds one); `ctx.footage` and `ctx.camera`, the recording's and the webcam's rects (x, y, w, h in uv; camera zero-sized when absent) — all already placed through zoom and layout; `fx_to_footage(uv)` and `fx_from_footage(p)` map between output uv and the recording's own surface (0..1 across the recording) through the warp the compositor draws it with, zoom, layout and 3D camera included, and `ctx.footage_aspect` is its width over height: the effect works in whichever of the two spaces fits it; `fx_px(n)`, n pixels of a 1080p frame as a fraction of the height; `fx_hash(vec2<u32>)`, a random number in [0,1) identical on every GPU. Rules: the effect is a pure function of those inputs (no state between frames), so the preview and the export show the same pixels; reason in uv or fx_px, never raw pixels; every loop is a `for` with literal bounds (`for (var i = 0; i < 16; i++)`), at most 4096 iterations per pixel — the one hard rule. `params` declares up to 8 (recommended) parameters, each `{type: number|toggle|color|point, name, label, default}` (number also `min`, `max`); declare one only when moving it visibly changes the effect over its whole range. `target`: `image` (default: the picture under the annotations and captions), `text` (the annotations and captions alone, premultiplied alpha), `both` (everything); use text or both only when asked. One effect runs at a time: to combine two looks, write them in one shader. The result gives the cost tier, in frame() and footage() reads per pixel, with what a 1080p frame pays on a fast discrete GPU (RTX 4070 Ti) and on an integrated one (Apple M1): light ≤ 8, up to 0.06 ms and 1.3 ms; medium ≤ 32, 0.14 ms and 4.3 ms; heavy ≤ 128, 0.54 ms and 16 ms; very heavy beyond, 2 ms and 49 ms at 512 reads. It also gives a recommendation when the tier runs long: light anywhere, medium up to a minute, heavy up to 10 s, very heavy a few seconds. It is a recommendation, never a refusal. Code that does not compile is refused with the compiler's error at the line of your code: fix it and call again. Check the result with getFrames.",
setEffect:
Expand Down
8 changes: 7 additions & 1 deletion electron/ai-edition/editor-settings-tool.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ describe("setEditorSettings", () => {
borderRadius: 12,
frame: "laptop",
audioGainDb: -3,
recordingMuted: true,
autoFocusAll: true,
cursor: { size: 2, theme: "glass-lens", show: true, asArrow: ["text"] },
});
Expand All @@ -80,14 +81,19 @@ describe("setEditorSettings", () => {
borderRadius: 12,
frame: "laptop",
audioGainDb: -3,
recordingMuted: true,
autoFocusAll: true,
cursorTheme: "glass-lens",
});
expect(settings.cursor.size).toBe(2);
expect(settings.cursor.asArrow).toEqual(["text"]);

const snapshot = read(document);
expect(snapshot.settings).toMatchObject({ padding: 20, aspectRatio: "auto" });
expect(snapshot.settings).toMatchObject({
padding: 20,
aspectRatio: "auto",
recordingMuted: true,
});
expect(snapshot.settings.cursor).toMatchObject({ size: 2, theme: "glass-lens" });
// What is read is what the tool takes: the whole block goes back in unchanged.
expect(set(document, snapshot.settings).ok).toBe(true);
Expand Down
2 changes: 2 additions & 0 deletions electron/ai-edition/editor-settings-tool.ts
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,7 @@ export const setEditorSettingsArgs = z
webcamWallpaper: wallpaperArg("webcamWallpaper"),
webcamBlurIntensity: bounded("webcamBlurIntensity"),
audioGainDb: z.number().min(-AUDIO_GAIN_DB_LIMIT).max(AUDIO_GAIN_DB_LIMIT),
recordingMuted: z.boolean(),
autoFocusAll: z.boolean(),
cursor: z
.object({
Expand Down Expand Up @@ -191,6 +192,7 @@ function settingsForModel(document: AxcutDocument) {
webcamWallpaper: elideDataUrl(s.webcamWallpaper),
webcamBlurIntensity: s.webcamBlurIntensity,
audioGainDb: s.audioGainDb,
recordingMuted: s.recordingMuted,
autoFocusAll: s.autoFocusAll,
cursor: {
show: s.cursorShow,
Expand Down
11 changes: 10 additions & 1 deletion src/cli/CliExportRunner.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,12 @@ import {
isAudioExportFormat,
} from "@/lib/cliContracts";
import { GIF_SIZE_PRESETS, type GifSizePreset } from "@/lib/exporter";
import { extractAudio, isolateAudioTrack } from "@/lib/exporter/audioExport";
import {
exportIsSilent,
extractAudio,
isolateAudioTrack,
removeAudioStream,
} from "@/lib/exporter/audioExport";
import { calculateMp4ExportSettings } from "@/lib/exporter/mp4ExportSettings";
import { outputFrameCount } from "@/lib/exporter/outputFrameCount";
import { mixVoiceoverIntoVideo } from "@/lib/exporter/voiceoverMix";
Expand Down Expand Up @@ -215,6 +220,10 @@ async function runExport(request: CliExportRequest): Promise<CliDoneResult> {
bitrate: outDims.bitrate,
});

if (!request.audioPath && exportIsSilent(axcutDocument, sceneDesc)) {
await removeAudioStream(outPath);
}

if (request.audioPath) {
window.electronAPI.cliProgress({ percentage: 100, phase: "mixing-voiceover" });
const [videoResponse, audioResponse] = await Promise.all([
Expand Down
8 changes: 7 additions & 1 deletion src/components/ai-edition/ExportDialog.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ import {
MP4_FRAME_RATES,
type Mp4FrameRate,
} from "@/lib/exporter";
import { exportIsSilent, removeAudioStream } from "@/lib/exporter/audioExport";
import { calculateMp4ExportSettings, wouldUpscale } from "@/lib/exporter/mp4ExportSettings";
import { outputFrameCount } from "@/lib/exporter/outputFrameCount";
import {
Expand Down Expand Up @@ -115,6 +116,7 @@ function revealExportedFile(filePath: string): void {
* no camera — the no-webcam layout is a later step) and its source trim. */
function buildNativeClipList(document: AxcutDocument): CompositorClipInput[] {
const assetById = new Map(document.assets.map((a) => [a.id, a]));
const hasAudio = !getEditorSettings(document).recordingMuted;
return resolveVisibleClips(document).flatMap((clip) => {
const asset = assetById.get(clip.assetId);
if (!asset?.originalPath) {
Expand All @@ -135,7 +137,7 @@ function buildNativeClipList(document: AxcutDocument): CompositorClipInput[] {
sourceStartSec: clip.sourceStartSec,
sourceEndSec,
webcamOffsetSec: camera.offsetSec,
hasAudio: true,
hasAudio,
},
];
});
Expand Down Expand Up @@ -459,6 +461,10 @@ export function ExportDialog({ open, onClose, document }: ExportDialogProps) {
job.id,
);
if (activeExport.current !== job) return;
if (format !== "gif" && exportIsSilent(document, sceneDesc)) {
await removeAudioStream(pickedPath);
if (activeExport.current !== job) return;
}
setSavedPath(pickedPath);
setPhase("done");
// Reported after the export succeeded, so a cancelled or failed run never counts.
Expand Down
11 changes: 10 additions & 1 deletion src/components/ai-edition/RightPanes.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -3464,11 +3464,20 @@ export function AudioPane() {
onCommit={() => void commit()}
/>
</div>
<div className={styles.paneRow}>
<span className={styles.label}>{ts("audio.muteRecording")}</span>
<Toggle
checked={settings.recordingMuted}
disabled={!hasDocument}
ariaLabel={ts("audio.muteRecording")}
onChange={(v) => void set({ recordingMuted: v })}
/>
</div>
<button
type="button"
className={styles.secondaryBtn}
disabled={!hasDocument}
onClick={() => void set({ audioGainDb: 0 })}
onClick={() => void set({ audioGainDb: 0, recordingMuted: false })}
>
{ts("audio.reset")}
</button>
Expand Down
10 changes: 10 additions & 0 deletions src/components/ai-edition/VirtualPreview.audio.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,16 @@ describe("applyPreviewAudioSettings", () => {
expect(element.volume).toBe(1);
});

it("silences a muted recording on its own gain, leaving the output gain to the tracks (#1104)", () => {
const graph = fakeGraph();
applyPreviewAudioSettings(graph, [], 0, Number.NEGATIVE_INFINITY);
expect(graph.voice.gain.value).toBe(0);
expect(graph.gain.gain.value).toBe(1);
const element = { volume: 1 } as HTMLAudioElement;
applyPreviewAudioSettings(null, [element], 0, Number.NEGATIVE_INFINITY);
expect(element.volume).toBe(0);
});

it("leaves the elements alone once the graph is carrying the gain", () => {
// Their audio no longer reaches the default output, so `volume` would only
// scale the signal a second time on its way into the node.
Expand Down
9 changes: 6 additions & 3 deletions src/components/ai-edition/VirtualPreview.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -681,9 +681,12 @@ export function VirtualPreview({
);
}
}, [voicePathsKey, retryToken, activeVoicePath]);
const voiceGainDb = activeSource?.filePath
? (loudnessGainDbByPath.get(activeSource.filePath) ?? 0)
: 0;
// A muted recording is silenced on its own gain, so the added tracks keep playing (#1104).
const voiceGainDb = settings.recordingMuted
? Number.NEGATIVE_INFINITY
: activeSource?.filePath
? (loudnessGainDbByPath.get(activeSource.filePath) ?? 0)
: 0;
const voiceGainDbRef = useRef(voiceGainDb);
voiceGainDbRef.current = voiceGainDb;

Expand Down
2 changes: 1 addition & 1 deletion src/components/ai-edition/v4/V4Timeline.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -2642,7 +2642,7 @@ export function V4Timeline({
assetDurationSec={asset?.durationSec}
sourceStartSec={c.sourceStartSec}
sourceEndSec={c.sourceEndSec ?? c.sourceStartSec + dur}
gain={audioGainScalar(settings.audioGainDb)}
gain={settings.recordingMuted ? 0 : audioGainScalar(settings.audioGainDb)}
/>
<div className={styles.tlClipLabel}>
<button
Expand Down
1 change: 1 addition & 0 deletions src/i18n/locales/ar/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,7 @@
},
"audio": {
"outputGain": "ضبط مستوى الإخراج",
"muteRecording": "كتم صوت التسجيل",
"help": "تُضبط الأصوات تلقائيًا على مستوى جهارة قياسي. اضبط مستوى الإخراج انطلاقًا منه؛ يُطبَّق بالطريقة نفسها في المعاينة وعند التصدير.",
"reset": "إعادة ضبط الصوت",
"title": "الصوت"
Expand Down
1 change: 1 addition & 0 deletions src/i18n/locales/cs/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,7 @@
},
"audio": {
"outputGain": "Výstupní úroveň",
"muteRecording": "Ztlumit nahrávku",
"help": "Hlas se automaticky vyrovná na standardní hlasitost. Výstupní úroveň upravte od ní; aplikuje se stejně v náhledu i při exportu.",
"reset": "Obnovit zvuk",
"title": "Zvuk"
Expand Down
1 change: 1 addition & 0 deletions src/i18n/locales/de/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,7 @@
},
"audio": {
"outputGain": "Ausgangspegel",
"muteRecording": "Aufnahme stummschalten",
"help": "Stimmen werden automatisch auf eine Standardlautheit gebracht. Passe den Ausgangspegel davon ausgehend an; er gilt in der Vorschau und im Export identisch.",
"reset": "Audio zurücksetzen",
"title": "Audio"
Expand Down
1 change: 1 addition & 0 deletions src/i18n/locales/en/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,7 @@
},
"audio": {
"outputGain": "Output level",
"muteRecording": "Mute recording",
"help": "Voices are levelled to a standard loudness automatically. Adjust the output level from there; it applies identically in the preview and the export.",
"reset": "Reset audio",
"title": "Audio"
Expand Down
1 change: 1 addition & 0 deletions src/i18n/locales/es/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,7 @@
},
"audio": {
"outputGain": "Ajuste de salida",
"muteRecording": "Silenciar la grabación",
"help": "Las voces se nivelan automáticamente a una sonoridad estándar. Ajusta el nivel de salida a partir de ahí; se aplica igual en la vista previa y en la exportación.",
"reset": "Restablecer audio",
"title": "Audio"
Expand Down
1 change: 1 addition & 0 deletions src/i18n/locales/fr/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,7 @@
},
"audio": {
"outputGain": "Niveau de sortie",
"muteRecording": "Couper le son de l'enregistrement",
"help": "Les voix sont automatiquement ramenées à une sonie standard. Ajustez le niveau de sortie à partir de là ; il s’applique à l’identique dans l’aperçu et à l’export.",
"reset": "Réinitialiser l’audio",
"title": "Audio"
Expand Down
1 change: 1 addition & 0 deletions src/i18n/locales/it/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,7 @@
},
"audio": {
"outputGain": "Livello di uscita",
"muteRecording": "Disattiva audio della registrazione",
"help": "Le voci vengono livellate automaticamente a un volume percepito standard. Regola il livello di uscita a partire da lì; si applica allo stesso modo nell’anteprima e nell’esportazione.",
"reset": "Reimposta audio",
"title": "Audio"
Expand Down
1 change: 1 addition & 0 deletions src/i18n/locales/ja-JP/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,7 @@
},
"audio": {
"outputGain": "出力レベル",
"muteRecording": "録画の音声をミュート",
"help": "音声は標準のラウドネスに自動で揃えられます。そこから出力レベルを調整してください。プレビューと書き出しで同じように適用されます。",
"reset": "オーディオをリセット",
"title": "オーディオ"
Expand Down
1 change: 1 addition & 0 deletions src/i18n/locales/ko-KR/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,7 @@
},
"audio": {
"outputGain": "출력 레벨",
"muteRecording": "녹화 오디오 음소거",
"help": "음성은 표준 라우드니스로 자동 조정됩니다. 거기서부터 출력 레벨을 조정하세요. 미리보기와 내보내기에 동일하게 적용됩니다.",
"reset": "오디오 재설정",
"title": "오디오"
Expand Down
1 change: 1 addition & 0 deletions src/i18n/locales/pt-BR/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,7 @@
},
"audio": {
"outputGain": "Nível de saída",
"muteRecording": "Silenciar a gravação",
"help": "As vozes são niveladas automaticamente a uma loudness padrão. Ajuste o nível de saída a partir daí; ele se aplica da mesma forma na prévia e na exportação.",
"reset": "Redefinir áudio",
"title": "Áudio"
Expand Down
1 change: 1 addition & 0 deletions src/i18n/locales/ru/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,7 @@
},
"audio": {
"outputGain": "Уровень выхода",
"muteRecording": "Отключить звук записи",
"help": "Голос автоматически выравнивается до стандартной громкости. Настройте уровень выхода относительно неё; он одинаково применяется в предпросмотре и при экспорте.",
"reset": "Сбросить аудио",
"title": "Аудио"
Expand Down
1 change: 1 addition & 0 deletions src/i18n/locales/tr/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,7 @@
},
"audio": {
"outputGain": "Çıkış seviyesi",
"muteRecording": "Kaydın sesini kapat",
"help": "Sesler otomatik olarak standart bir ses yüksekliğine getirilir. Çıkış seviyesini buradan ayarlayın; önizlemede ve dışa aktarmada aynı şekilde uygulanır.",
"reset": "Sesi sıfırla",
"title": "Ses"
Expand Down
1 change: 1 addition & 0 deletions src/i18n/locales/vi/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,7 @@
},
"audio": {
"outputGain": "Mức đầu ra",
"muteRecording": "Tắt tiếng bản ghi",
"help": "Giọng nói được tự động cân bằng về độ lớn tiêu chuẩn. Hãy điều chỉnh mức đầu ra từ đó; nó được áp dụng giống hệt nhau trong bản xem trước và khi xuất.",
"reset": "Đặt lại âm thanh",
"title": "Âm thanh"
Expand Down
1 change: 1 addition & 0 deletions src/i18n/locales/zh-CN/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,7 @@
},
"audio": {
"outputGain": "输出电平",
"muteRecording": "静音录制音频",
"help": "人声会自动统一到标准响度。在此基础上调整输出电平;它在预览和导出中的效果完全一致。",
"reset": "重置音频",
"title": "音频"
Expand Down
1 change: 1 addition & 0 deletions src/i18n/locales/zh-TW/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,7 @@
},
"audio": {
"outputGain": "輸出音量",
"muteRecording": "將錄影靜音",
"help": "人聲會自動統一到標準響度。在此基礎上調整輸出音量;它在預覽與匯出中的效果完全一致。",
"reset": "重設音訊",
"title": "音訊"
Expand Down
Loading
Loading