feat(parakeet-cpp): VAD (Silero and the Moondream head), vad_model option and gallery entries - #12463
Merged
Merged
Conversation
The backend now serves the VAD gRPC call (POST /vad and /v1/vad) with the standalone VAD of libparakeet. It accepts a Silero VAD GGUF as the model file, or an ASR model with a VAD head (Moondream Ultra and Redux). The request audio is float32 PCM at 16 kHz; the response lists the speech segments in seconds, like the silero-vad backend. A model with neither fails the request with the library message. The vad_threshold, vad_min_pause, vad_min_speech, vad_speech_pad and vad_max_segment options tune the segmenter. Unset values keep the defaults of the detector in use, and a bad value fails the load. The vad_model option names a Silero GGUF, resolved against the models directory like the other companion files. It lets any ASR model cut long audio at pauses through parakeet_capi_transcribe_path_json_vad_with, and it implies vad. vad:true alone still uses the model's own head. The new symbols are probed like the existing optional ones. A library without them still loads; the feature that needs one fails with a clear message only when it is used. Assisted-by: Claude:claude-sonnet-5-5 [go test]
Add VAD-only entries for the parakeet-cpp backend: the VAD heads of Moondream Redux (packed, CPU) and Ultra (Q8_0), which share their files with the existing ASR entries, and Silero VAD v6.2.3 as a GGUF (MIT, Silero Team). The parakeet-cpp-vad entry installs Silero; it has no variants, because variant ranking prefers the larger build that fits and these are different detectors. Add parakeet-cpp-tdt-0.6b-v3-silero-vad, a v3 entry that sets vad_model so long audio is cut at pauses by Silero. The Silero GGUF entries point at the intended Hugging Face URL of the file; the existing silero-vad entries are unchanged. A test checks the usecases, the shared files and the default entry and the vad_model reference. Assisted-by: Claude:claude-sonnet-5-5 [go test]
Document the VAD endpoint on the parakeet-cpp backend (Silero GGUF and the VAD heads of Moondream Ultra and Redux), the vad_* tuning options, and the vad_model option that lets an ASR model without a VAD head cut long audio with Silero. Assisted-by: Claude:claude-sonnet-5-5
Pin the release that adds the standalone VAD (Ultra/Redux head and Silero) and the C API calls the backend now uses. Assisted-by: Claude:claude-sonnet-5-5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The
parakeet-cppbackend now serves/vadand/v1/vadwith the standalone VAD of parakeet.cpp: Silero VAD v6.2.3 (MIT) and the VAD head of Moondream Ultra and Redux. It also gets avad_modeloption and gallery entries for all of these.What changed
vad_model:<silero gguf>option. Any parakeet ASR model without a VAD head (v3, Nemotron) can use it to cut long audio at pauses with Silero.vad_threshold,vad_min_pause,vad_min_speech,vad_speech_pad,vad_max_segment. The existingsilero-vadbackend uses ms-based options with different names, so the two are not interchangeable.parakeet-cpp-silero-vad-f16,parakeet-cpp-vad-moondream-redux-packed,parakeet-cpp-vad-moondream-ultra-q8_0,parakeet-cpp-vad(installs Silero) andparakeet-cpp-tdt-0.6b-v3-silero-vad.docs/content/.Tests
go test ./backend/go/parakeet-cpp/...: pass (Go specs with stubbed C calls)go test ./core/gallery/...: passgolangci-lint runonbackend/go/parakeet-cppandcore/gallery: 0 issues/v1/vadsegments equalparakeet-cli vadoutput (max difference 0 s; Silero 18 segments, Redux 16, on a 60 s clip). v3 plus Silero transcription equals the CLI output.Not verified
/v1/vad./v1/vadneeds a smaller payload or a higher limit.go vetreports anunsafe.Pointerwarning ingoparakeetcpp.go. It predates this branch.Notes for Reviewers
Commits carry
Assisted-bytrailers and noSigned-off-by, per the AI contribution policy. The submitter adds sign-off.Signed commits
🤖 Generated with Claude Code