Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 36 additions & 44 deletions fern/customization/voice-pipeline-configuration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -217,8 +217,8 @@ Uses AI models to analyze speech patterns, context, and audio cues to predict wh
- **livekit**: Advanced model trained on conversation data (English only)
- **vapi**: VAPI-trained model (non-English conversations or LiveKit alternative)

**Audio-based providers:**
- **krisp**: Audio-based model analyzing prosodic features (intonation, pitch, rhythm)
**Custom providers:**
- **custom-endpointing-model**: Sends each endpointing decision to your own server, which returns a `timeoutSeconds` value

**Audio-text based providers:**
- **deepgram-flux**: Deepgram's latest transcriber model with built-in conversational speech recognition. Use `flux-general-en` for English-only conversations or `flux-general-multi` for multilingual conversations.
Expand All @@ -235,7 +235,7 @@ Uses AI models to analyze speech patterns, context, and audio cues to predict wh
- **AssemblyAI**: Use when AssemblyAI is already your transcriber provider and you want integrated end-of-turn detection
- **LiveKit**: English conversations where Deepgram is not the transcriber of choice.
- **Vapi**: Non-English conversations with default stop speaking plan settings
- **Krisp**: Non-English conversations with a robustly configured stop speaking plan
- **Custom endpointing model**: When your own server should decide the endpointing timeout for each turn

### Deepgram Flux configuration

Expand Down Expand Up @@ -357,31 +357,52 @@ The system continuously analyzes the latest user message and applies the first m
- Scenarios requiring predictable, rule-based endpointing behavior
- Fallback option when other smart endpointing providers aren't suitable

### Krisp threshold configuration
### Custom endpointing model configuration

Krisp's audio-base model returns a probability between 0 and 1, where 1 means the user definitely stopped speaking and 0 means they're still speaking.

**Threshold settings:**

- **0.0-0.3:** Very aggressive detection - responds quickly but may interrupt users mid-sentence
- **0.4-0.6:** Balanced detection (default: 0.5) - good balance between responsiveness and accuracy
- **0.7-1.0:** Conservative detection - waits longer to ensure users have finished speaking
Sends each endpointing decision to your own server instead of a built-in model. Vapi POSTs the current transcript to `server.url`; your server responds with a `timeoutSeconds` value, the number of seconds to wait before considering the user's turn finished. The timeout resets each time a new transcript is received.

**Configuration example:**

```json
{
"startSpeakingPlan": {
"smartEndpointingPlan": {
"provider": "krisp",
"threshold": 0.5
"provider": "custom-endpointing-model",
"server": {
"url": "https://your-server.com/endpointing"
}
}
}
}
```

**Important considerations:**
Since Krisp is audio-based, it always notifies when the user is done speaking, even for brief acknowledgments. Configure the stop speaking plan with appropriate `acknowledgementPhrases` and `numWords` settings to handle backchanneling properly.
**Request sent to your server:**

```json
{
"message": {
"type": "call.endpointing.request",
"messages": [
{
"role": "user",
"message": "Hello, how are you?",
"time": 1234567890,
"secondsFromStart": 0
}
]
}
}
```

**Expected response:**

```json
{
"timeoutSeconds": 0.5
}
```

If `server` is not provided, the request is sent to `assistant.server`, then `org.server` if that isn't set either.

### Assembly turn detection

Expand Down Expand Up @@ -665,35 +686,6 @@ User Interrupts → Assistant Audio Stopped → backoffSeconds Blocks All Output

**Optimized for:** Text-based endpointing with longer timeouts for different speech patterns and international support.

### Audio-based endpointing (Krisp example)

```json
{
"startSpeakingPlan": {
"waitSeconds": 0.4,
"smartEndpointingPlan": {
"provider": "krisp",
"threshold": 0.5
}
},
"stopSpeakingPlan": {
"numWords": 2,
"voiceSeconds": 0.2,
"backoffSeconds": 1.0,
"acknowledgementPhrases": [
"okay",
"right",
"uh-huh",
"yeah",
"mm-hmm",
"got it"
]
}
}
```

**Optimized for:** Non-English conversations with robust backchanneling configuration to handle audio-based detection limitations.

### Audio-text based endpointing (Assembly example)

```json
Expand Down
4 changes: 4 additions & 0 deletions fern/phone-numbers/free-telephony.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ This guide details how to create free phone numbers on the Vapi platform, which
- **Free Vapi numbers are for US national use.** They cannot make international calls. For international use, [import a number from another provider](/phone-numbers/import-twilio).
- **Outbound calling not supported.** Free Vapi numbers are inbound only. Import a number from a supported telephony provider to make outbound calls. See [Outbound calling](/calls/outbound-calling).
- **Outbound Campaigns require an imported number.** Free Vapi numbers cannot be used to launch [Outbound Campaigns](/outbound-campaigns/quickstart).
- **Porting out is not supported.** A free Vapi number is provisioned on Vapi's own carrier accounts, not a carrier account you hold, so there is no account number or port-out PIN to request and the number cannot be ported to another carrier.
- **You are responsible for lawful use.** Obtain the required consent before making outbound calls and follow applicable calling, telemarketing, and do-not-call rules. See the [TCPA consent guide](/tcpa-consent) and [Vapi Terms of Service](https://vapi.ai/terms-of-service).

### Frequently Asked Questions
Expand All @@ -69,4 +70,7 @@ This guide details how to create free phone numbers on the Vapi platform, which
<Accordion title="Does free mean calls cost nothing?" icon="phone" iconType="regular">
No. The Vapi-managed phone number is free, but calls and other platform usage still consume credits and are billed according to Vapi pricing.
</Accordion>
<Accordion title="Can I port a free Vapi number out to my own carrier account?" icon="phone" iconType="regular">
No. Free Vapi numbers are provisioned on Vapi's own carrier accounts, so there is no account number or port-out PIN Vapi can provide. To manage a number under your own carrier account, buy or port it into that provider directly, then [import it into Vapi](/phone-numbers/import-twilio).
</Accordion>
</AccordionGroup>
Loading