VoiceThere cloud agents use the @node-webrtc-rust/sdk speech pipeline: voice activity detection (VAD), optional STT gating, barge-in during agent TTS, and structured speech events. Advanced settings let you override library defaults per project without changing your agent bundle.

Where to configure

Dashboard: Project Settings → Voice (STT / TTS) → expand Advanced voice pipeline (collapsed by default). Each field has an info tooltip with tuning guidance.
CLI: voicethere projects voice-advanced — see examples below.

Changes are saved on the project immediately but apply to live sessions only after you Deploy to cloud on the project overview.

Voice quality changes both the price and how fast replies start. Medium voices are the recommended default. See Choosing a voice quality.

Inbound noise suppression (RNNoise)

Voice and Voice+Data runners apply Xiph RNNoise to inbound PCM before VAD and STT. The toggle is on by default for new voice projects. Turn it off when CPU is tight (~8m per live inbound stream, roughly ~80m extra at ten concurrent sessions on a 1-core runner).

Dashboard: Project Settings → Voice (STT / TTS) → Advanced voice pipeline → Inbound noise suppression (RNNoise).
Data-only projects: the Voice panel (and this toggle) is hidden — data-only runners do not construct a voice pipeline.
CLI: voicethere projects voice-advanced set noiseSuppression.enabled false — applies on the next deploy.

Phrase cache

When your agent says the same thing twice, a greeting for example, the agent runner plays the second one from memory instead of synthesising it again. For repeats that takes the speech-generation delay down to almost nothing. The cache is on by default and applies to VoiceThere voices and cloud TTS vendors alike.

Replays are free. A reply played from memory uses no TTS credits, and with a cloud vendor it never reaches the vendor, so it is not on your vendor bill either. See Speech pricing.

The cache holds up to 16 MiB of audio per agent runner, about 87 seconds of speech (audio is kept as played: 48 kHz stereo, 16-bit). A single reply is cached only up to 2 MiB, about 11 seconds; longer replies are synthesised every time. There is no fixed expiry: when the cache is full, the least recently played reply is dropped, and a deploy starts empty. Only replies that were fully synthesised are kept, and entries are scoped to one project and one voice.

Turn it off when every reply has to be freshly synthesised, for instance because you prefer that no spoken text is kept in runner memory, or when you load test speech capacity and need each request to hit the synthesiser.

Dashboard: Advanced voice pipeline → TTS playback → Phrase cache.
CLI: voicethere projects voice-advanced set tts.phraseCache false. It applies on the next deploy.

Sentence start capitalization

Before synthesis, the agent's text is split into sentences. The voice engine treats a period followed by a lowercase word as an abbreviation inside one sentence, as in "approx. eight". So a reply such as Okay. eight one six is synthesized as one sentence, and its first audio starts later. On our speech servers the first audio chunk took about 250 ms instead of about 100 ms.

With auto(the default), the first letter after ". ", "! " or "? " is uppercased in the copy of the text handed to the voice engine, and only there. Your speak() text, speech events, transcripts, conversation history, agent logs and the phrase cache keep the text exactly as you sent it. Speech has no letter case, so the words sound the same. What changes is how the engine reads the period: as a sentence end, so the first audio of the reply starts sooner.

These cases are left alone:

Ellipses, as in "wait... then".
Abbreviations with inner dots, such as "e.g. this", "i.e. that" and "U.S. troops".
Mr. Mrs. Ms. Dr. St. vs. etc.
Decimals such as 1.5.
The very first letter of the text.

Set off if you rely on lowercase after a period for a reason: you write run-on text on purpose, or you speak codes or notation where a capital changes the meaning. The engine then gets your text exactly as written. The setting applies to VoiceThere speech (cluster Sherpa and local Sherpa TTS) and takes effect on the next deploy.

To see what the engine received, turn on the TTS text trace. Every agent_speaking_start speech event also carries sentenceCaseAdjusted, the number of letters that were capitalized for that reply.

Dashboard: Advanced voice pipeline → TTS playback → Sentence start capitalization.
CLI: voicethere projects voice-advanced set tts.sentenceCase off

TTS text trace

With the trace on, your project's agent logs get one line for each agent reply. The line holds the text you sent and the text the voice engine received. Use it to see what sentence start capitalization changed, and to check any later rule that adjusts the text before synthesis.

It switches itself off after 24 hours. Turning it on again restarts the 24 hours. The runner stops writing at the end time, so you do not need to deploy again to turn it off.
It applies on the next deploy, like the other advanced settings.
The text goes only to your project's agent logs. It is not written to VoiceThere platform logs.
Log volume: one extra line per reply while it is on.
Always available: the agent_speaking_start speech event has a sentenceCaseAdjusted field, the count of letters capitalized for that reply. It is present for VoiceThere speech and does not need the trace.

Dashboard: Advanced voice pipeline → TTS playback → TTS text trace (24 h). While it is on, the switch shows when it ends.
CLI: voicethere projects voice-advanced set debug.ttsTextTrace on, and off to stop it early.

Speech pad

vad.speechPadMs is how much audio from before the detected start of speech is kept and fed to speech-to-text, so the first word is not clipped. The product default is 1000 ms.

We measured this in load tests with 45 and 30 simultaneous conversations. With 1000 ms, about 64 % of transcripts started with a lowercase word. With 1500 ms it was about 9 %. The recognizer more often treats the first word as the start of a sentence when it hears more silence before it. Word accuracy was the same in both runs, and no first or last words were clipped.

A longer pad sends a little more audio per turn to speech-to-text. It does not change how fast the final transcript arrives. A shorter pad risks clipping a quick first word. Lowercase transcripts only matter if your agent echoes them or reads casing. With tts.sentenceCase on auto, the spoken reply is not affected.

Dashboard: Advanced voice pipeline → VAD → Speech pre-roll pad (ms).
CLI: voicethere projects voice-advanced set vad.speechPadMs 1500

Reconnect window

When a caller's connection drops, the agent runner keeps the conversation open for a while so the client can reconnect and carry on. The reconnect window sets how long that is: 15 to 30 seconds, default 15. The browser client follows the project's window automatically.

Noticing the drop and reconnecting take about five seconds of the window, so a window of N seconds rides out a network outage of roughly N minus 5 seconds: about 10 seconds with the default, about 25 seconds at the maximum.

Billing: if the connection comes back within the window, the conversation continues and the whole call is billed, including the interruption, so a longer window means more recovered (and billed) conversations. If it does not come back, billing stops at the moment the connection dropped. The wait itself is not billed, but the dropped session holds a runner slot for the whole window.

Dashboard: Advanced voice pipeline → Connection → Reconnect window (s).
CLI: voicethere projects voice-advanced set connection.reconnectWindowSec 30. It applies on the next deploy.

Pipeline overview

VAD detects when the user is speaking on the inbound audio track.
When gateStt is enabled (recommended), STT receives audio only while the gate is open — during speech, pending lead-in, and post-speech hold.
Barge-in stops agent TTS when the user interrupts. With requireSttPartial (default), playback continues until STT returns a qualifying partial (semantic interrupt).
Your agent receives user_speech_final and other speech events, then runs LLM/TTS for the reply.

How the end of a turn is decided

Your agent receives each caller sentence as one user_speech_final. Deciding when that sentence is over takes two waits, one after the other. Both count audio as it arrives (20 ms per frame), not clock time.

The caller starts talking. After vad.minSpeechDurationMs (250 ms) of sound, VAD declares speech and you get user_speaking_start. STT also receives the audio from just before that moment (vad.speechPadMs, 1000 ms), so the first word is not clipped. This pad only affects the start of an utterance, never its end.
While they talk, STT gets every frame and returns partial transcripts.
Silence window. When they stop, VAD counts silent audio. Any speech resets the count, so the short pauses between words do not end the turn. After vad.minSilenceDurationMs (1100 ms) of unbroken silence, VAD decides the caller has stopped. Nothing is sent to your agent yet.
STT gate hold. For another vad.sttGateHoldMs (400 ms), STT keeps receiving audio, so it can finish a quiet last word that VAD already counted as silence. If the caller speaks again before 90% of the hold has passed, the hold is cancelled and the new words join the same utterance. Later than that, the earlier sentence is closed and the new speech starts a new turn.
Final. When the hold ends, STT returns its final transcript, or one is built from the last partial after vad.utteranceFinalizeTimeoutMs (1500 ms). Your agent receives user_speech_final and user_speaking_end together and can reply.

In parallel, if VAD heard speech but STT returns no partial within vad.sttListenTimeoutMs (4000 ms), the turn ends with user_stt_not_found. That usually means the caller made a noise rather than speaking, or STT is falling behind.

Why two waits

They answer different questions. The silence window asks whether the caller has stopped making sound. The gate hold asks whether STT has heard everything, and whether the caller is really done or only pausing. One short window would cut callers off mid-sentence and drop last words. One long window would slow every reply and still stop feeding STT at the wrong moment.

What callers notice

With the defaults, the time from a caller's last word to the agent's first sound is 1.1 s of silence plus 0.4 s of hold, then the STT final and your agent's own reply time. Most of it is deliberate waiting. Shorter values make replies faster, at the cost of cutting off people who pause while they think.

End-of-turn wait

vad.minSilenceDurationMs (Minimum silence duration) and vad.sttGateHoldMs(STT gate hold) together set how long the agent waits after the caller stops before it answers. The defaults are 1100 ms and 400 ms. Before 2026-10-18 they were 1300 ms and 1000 ms. Projects that set their own values keep them. Projects on the defaults use the new ones. You can change both in the project's voice settings (Advanced voice pipeline) or with the CLI.

Shorter values answer faster, but they also split a sentence in two when the caller pauses mid-sentence: the first half becomes one turn and the second half another. The table is from a staging test where each sentence was spoken in two halves with a fixed pause between them. The wait is the median time from the last speech to the final transcript in a single conversation.

Minimum silence / gate hold (ms)	Pauses that stay one turn	Wait (median)
1300 / 1000 (old default)	300 and 500 ms; 700 ms splits	1.19 s
1100 / 400 (default)	300 ms; 500 ms splits	0.80 s
800 / 400	None; 300 ms already splits	0.69 s

Lower both for short commands and quiet rooms. Raise them if callers think while they talk and get cut off.

Tuning

Goal	Change	Trade-off
Faster replies (short answers, quiet rooms)	vad.minSilenceDurationMs 800–1000 (the gate hold is already short)	Callers who pause mid-sentence get cut off; finals can lose the last word
Callers who think while talking (support, dictation)	vad.minSilenceDurationMs 1300–2000 and vad.sttGateHoldMs 1000	Slower replies
Last word missing from finals	Raise vad.sttGateHoldMs	Slower replies
First word clipped	Raise vad.speechPadMs	No effect on reply time
Slow or remote STT	Raise vad.sttListenTimeoutMs	A noise-only utterance takes longer to be discarded

Spoken language identification

Cloud voice deploys with VoiceThere STT run Whisper tiny for spoken-language ID and emit user_language events. Defaults: enabled, languageId.minSpeechMs 1500 (cloud product default, not the SDK 1000 ms default).

Dashboard: Advanced voice pipeline → Spoken language identification.
CLI: voicethere projects voice-advanced set languageId.enabled false or voicethere projects voice-advanced set languageId.minSpeechMs 3000.

See Spoken language identification for detection tuning and Switching language mid-conversation for languageId.autoSwitch.* (including the wait message settings) and voice.profilesByLanguage. The wait message mode immediate plays at the end of the utterance if your runner does not yet support protected playback. Client events (user_speech_final_held, language_id_skipped, voice_language_switching, …) are listed on that page; the matching agent-log lines are on System log events.

languageId.timing decides when language ID runs: early (default, once as soon as languageId.minSpeechMs of speech is buffered, while the caller is still talking), end_of_utterance (once, after the caller stops) or continuous (repeated passes while the caller talks). With auto-switch on, the runner holds each final until language ID decides, capped by languageId.autoSwitch.finalHoldMs. With early the decision is ready before the final for any utterance longer than minSpeechMs, so the hold adds no delay. Utterances shorter than minSpeechMs skip language ID and their final is released immediately. continuous costs the most CPU and can slow VoiceThere TTS on busy runners; if agent audio gets choppy, switch back with voicethere projects voice-advanced set languageId.timing early.

CLI examples

# List resolved values for the linked project
voicethere projects voice-advanced list

# Disable semantic barge-in (instant VAD interrupt — noisier)
voicethere projects voice-advanced set vad.bargeIn.requireSttPartial false

# Faster end-of-utterance in quiet rooms (splits pauses of about 300 ms)
voicethere projects voice-advanced set vad.minSilenceDurationMs 800

# Back to the previous defaults, for callers who pause mid-sentence
voicethere projects voice-advanced set vad.minSilenceDurationMs 1300
voicethere projects voice-advanced set vad.sttGateHoldMs 1000

# Slightly slower Sherpa Piper TTS (clearer remote STT)
voicethere projects voice-advanced set tts.speed 0.9

# Synthesise every reply fresh instead of replaying repeats from memory
voicethere projects voice-advanced set tts.phraseCache false

# Hand the voice engine your text exactly as written (no capital after a period)
voicethere projects voice-advanced set tts.sentenceCase off

# Log the original and engine-input text of each reply for 24 hours
voicethere projects voice-advanced set debug.ttsTextTrace on

# Keep more audio before detected speech (fewer lowercase first words in transcripts)
voicethere projects voice-advanced set vad.speechPadMs 1500

# Hold a dropped conversation for 30 s so the client can reconnect (15–30, default 15)
voicethere projects voice-advanced set connection.reconnectWindowSec 30

# Disable inbound RNNoise (saves CPU on noisy multi-session pods)
voicethere projects voice-advanced set noiseSuppression.enabled false

# Disable spoken language ID (keeps Whisper model on runner)
voicethere projects voice-advanced set languageId.enabled false

# Wait longer before first user_language (1000–5000 ms; cloud default 1500)
voicethere projects voice-advanced set languageId.minSpeechMs 3500

# Restore library defaults
voicethere projects voice-advanced reset

Setting reference

Defaults match the SDK production preset (VOICE_AGENT_VAD_PRESET) plus events.mode: both.

Key	Default	When to change
voice_advanced.vad.enabled	true	Master switch for voice activity detection on inbound audio. Disable only for always-on STT experiments. Most voice agents should leave this on.
voice_advanced.vad.provider	energy	energy = RMS level detector (default build). We can enable silero = neural VAD, however this will require a new dedicated build
voice_advanced.vad.threshold	0.15	Energy VAD: RMS level (~0.05–0.2 typical). Raise in noisy rooms to reduce false speech starts. Lower if users must speak quietly and VAD misses them.
voice_advanced.vad.minSpeechDurationMs	250	Voiced audio must exceed this before user_speaking_start fires. Increase to ignore brief coughs and clicks. Decrease for snappier turn detection in quiet environments.
voice_advanced.vad.minSilenceDurationMs	1100	Unbroken silence after speech before VAD decides the caller stopped. Speech resets the count, so pauses between words do not end the turn. The STT gate hold runs after it. Lower (800–1000) for faster replies with short answers in quiet rooms. Raise (1300–2000) if callers pause mid-sentence and get cut off.
voice_advanced.vad.speechPadMs	1000	Audio from just before speech was detected that STT receives at speech start. Affects the start of an utterance only, not when it ends. Raise if the first syllable is clipped, especially during barge-in over agent TTS.
voice_advanced.vad.sampleRate	16000	Internal VAD sample rate. WebRTC PCM is resampled to mono 16 kHz for STT. Leave at 16000 unless you have a specific 8 kHz telephony pipeline.
voice_advanced.vad.gateStt	true	When true, STT receives audio only while the gate is open (speech, hold, or pending). Recommended for voice agents. Disable for continuous STT streaming (higher cost/noise).
voice_advanced.vad.gateSttOpenOnPending	true	When gateStt is true, feed STT during VAD pending speech before SpeechStart. Keep enabled to capture WebRTC lead-in audio. Disable to tighten when STT opens.
voice_advanced.vad.sttGateHoldMs	400	After the silence window, keep sending audio to STT this long so it can finish the last word. Speech before 90% of the hold continues the same utterance. Raise (up to 1000) if finals drop the last word or callers pause mid-sentence and get split into two turns. Lower for a faster user_speech_final; watch for truncated endings.
voice_advanced.vad.sttListenTimeoutMs	4000	After vad_triggered, emit user_stt_not_found when no STT partial arrives within this window. Raise in slow STT setups. Lower to fail fast when the user only made noise without speech.
voice_advanced.vad.utteranceFinalizeTimeoutMs	1500	Grace after the last partial or VAD SpeechEnd before forcing user_speech_final. Raise if cloud STT stalls on finals. Lower when partials are stable and you want snappier turns.
voice_advanced.vad.bargeIn.enabled	true	Master switch to stop agent TTS and emit barge_in when interrupted. Disable for broadcast-only agents where users must listen to the full reply.
voice_advanced.vad.bargeIn.useVad	true	When true, inbound VAD SpeechStart can trigger barge-in during agent TTS. Set false to allow manual barge-in via flushTts only (no automatic interrupt on noise).
voice_advanced.vad.bargeIn.flushTts	true	Clear pending outbound TTS PCM when barge-in runs. Disable only if your app manages playback cancellation itself.
voice_advanced.vad.bargeIn.requireSttPartial	true	During agent TTS, wait for a qualifying user_speech_partial before barge-in (semantic interrupt). Keep on to ignore coughs and tones. Set false for instant VAD barge (noisier, faster).
voice_advanced.vad.bargeIn.minSttPartialChars	2	Minimum trimmed STT partial length to trigger barge when requireSttPartial is true. Raise to ignore short false partials from echo or background speech.
voice_advanced.vad.bargeIn.agentPlaybackGuardMs	0	Ignore VAD barge-in for this many ms after agent TTS starts (mitigates speaker echo). Prefer headphones and requireSttPartial first. Raise only if speaker bleed falsely interrupts TTS.
voice_advanced.tts.speed	0.9	Sherpa Piper speaking-rate multiplier for VoiceThere TTS (1.0 = model default pace). Absolute pace is voice-dependent (e.g. Lessac sounds brisker than Amy at the same value) — retune after changing TTS voice. Lower for clearer remote STT under load (0.7–0.9). Raise toward 2.0 for faster replies. Applies on next deploy.
voice_advanced.tts.postUtteranceSilenceMs	2550	Real-time silent PCM streamed on agent outbound audio after each TTS utterance completes so remote listeners can finalize STT. Default aligns with VAD gate hold + min silence + 250 ms. Set 0 to disable (debug only).
voice_advanced.tts.phraseCache	true	Runner TTS phrase cache: repeated replies are served from memory on the runner instead of being synthesised again. On by default. Turn off when every reply must be synthesised fresh (privacy preference, load testing speech capacity). Applies to VoiceThere speech; see /docs/advanced-voice-settings#phrase-cache.
voice_advanced.tts.sentenceCase	auto	The voice engine reads a period followed by a lowercase word as an abbreviation. With auto, the first letter after a sentence end is capitalized in the text sent to the engine, so the engine reads it as a sentence end and the reply starts sooner. Your speak() text, transcripts and logs are not changed. Set off if you rely on lowercase after a period. See /docs/advanced-voice-settings#sentence-case.
voice_advanced.debug.ttsTextTrace	false	Writes one line per agent reply to your agent logs: the text you sent and the text the voice engine received. Switches itself off 24 hours after you turn it on. Use it to check sentence start capitalization. Turning it on again restarts the 24 hours. Applies on the next deploy and adds one log line per reply while on. See /docs/advanced-voice-settings#tts-text-trace.
voice_advanced.connection.reconnectWindowSec	15	How long a dropped voice conversation is held so the client can reconnect, 15 to 30 seconds. Default 15. If the connection comes back within the window, the conversation continues and the whole call is billed, including the interruption; a longer window means more recovered (and billed) conversations. If it does not come back, billing stops at the moment the connection dropped and the wait itself is not billed, but the dropped session holds a runner slot for the window. See /docs/advanced-voice-settings#reconnect-window.
voice_advanced.noiseSuppression.enabled	true	Applies Xiph RNNoise to inbound PCM before VAD and STT on voice runners. Default on for Voice and Voice+Data projects; hidden on data-only runners. Turn off if CPU is tight (~8m per live inbound stream, ~80m at 10 sessions).
voice_advanced.languageId.enabled	true	Detect the caller’s spoken language with Sherpa Whisper tiny and emit user_language events. Enabled automatically on cloud voice deploys with VoiceThere STT. Turn off to skip LID while keeping the model on the runner.
voice_advanced.languageId.minSpeechMs	1500	Inbound PCM duration before the first spoken-language identify each utterance (1000–5000 ms). Cloud default 1500; SDK/local default is 1000 when unset. Lower (toward 1000): first user_language sooner; Whisper tiny may mis-guess short audio. Higher (toward 5000): more accurate, slower. Utterances shorter than this skip language ID and their final is released immediately.
voice_advanced.languageId.timing	early	early: identify once as soon as the minimum speech is buffered, while the caller is still talking. end_of_utterance: identify once after the caller stops. continuous: identify repeatedly while the caller talks. early is ready before most finals, so the language-check hold usually adds no delay. end_of_utterance holds each final for one identify pass. continuous costs the most CPU; turn it off if agent audio gets choppy.
voice_advanced.languageId.autoSwitch.enabled	false	When spoken language ID disagrees with the session language, swap STT/TTS (and optionally replay the utterance). Off by default. Requires spoken language identification. Your agent can still call setVoiceLanguage without this. See /docs/voice-language-switch.
voice_advanced.languageId.autoSwitch.replayLastUtterance	true	Hold the revealing user_speech_final and replay inbound PCM after the STT pool swap. Disable if you prefer to drop the first final after a language flip.
voice_advanced.languageId.autoSwitch.finalHoldMs	5000	Safety cap: the runner holds a final until language ID decides; released after this many ms if LID has not answered. Raise only if LID routinely answers later than the cap on noisy lines.
voice_advanced.languageId.autoSwitch.minDwellMs	10000	Ignore further LID language flips for this long after a committed auto-switch. Prevents ping-pong when callers mix languages in one session.
voice_advanced.languageId.autoSwitch.confirmUtterances	1	Consecutive user_language detections required before auto-switch commits. Use 2–3 when short false LID guesses are common.
voice_advanced.languageId.autoSwitch.switchVoice	true	When auto-switch runs, change TTS as well as STT (off = STT-only). Turn off to keep the boot TTS voice while STT follows LID.
voice_advanced.languageId.autoSwitch.waitAudio	buffer_replay	buffer_replay: keep the caller's speech during the wait and replay it into the new STT. first_utterance: replay only the utterance that triggered language ID. Use first_utterance if you do not want speech from the wait window to reach your agent. See /docs/voice-language-switch.
voice_advanced.languageId.autoSwitch.waitMessage.mode	end_of_utterance	end_of_utterance: no, wait until the caller stops talking (default). immediate: yes, play a protected wait message right away, even if it cuts the caller off. off: no wait message. The message only plays when the target language is not already running. Agent-driven switches (setVoiceLanguage) never get a runner message.
voice_advanced.languageId.autoSwitch.waitMessage.skipWhenReady	true	Do not play the wait message when the target language is already running and the swap is instant. Turn off only if you want the message on every switch, for example to test it.
voice_advanced.languageId.autoSwitch.waitMessage.texts	{"de":"Einen Moment bitte, ich wechsle auf {language}.","en":"One moment, switching to {language}.","es":"Un momento, por favor, cambio al {language}.","fr":"Un instant, je passe en {language}.","it":"Un momento, cambio lingua: {language}.","nl":"Een moment, ik schakel over naar het {language}.","pl":"Chwileczkę, przełączam na język: {language}.","pt":"Um momento, vou mudar para {language}.","ru":"Одну минуту, переключаюсь на {language}."}	JSON object keyed by ISO 639-1 code. The text for the language being left is spoken with that language's voice. Up to 300 characters each. {language} is replaced with the name of the language being switched to, in the caller's current language. Example: {"en":"One moment, switching to {language}."}. Languages without a text fall back to the built-in text, then English.
voice_advanced.languageId.autoSwitch.readyMessage.minSwitchMs	2000	When the target language was already running at the end of the caller's turn but the switch took at least this long, the agent says a short message in the new voice before it answers. 0 turns the message off. Switches faster than this stay silent. See /docs/voice-language-switch#messages.
voice_advanced.languageId.autoSwitch.readyMessage.texts	{}	JSON object keyed by ISO 639-1 code. The text for the new language is spoken with the new voice. Up to 300 characters each. Example: {"de":"Okay, machen wir auf Deutsch weiter."}. Languages without a text use the built-in text, then English.
voice_advanced.voice.allowedLanguages		Languages permitted for setVoiceLanguage and LID auto-switch. Empty uses the boot language plus the languages in the per-language models table. Use * to allow any code for agent-driven switches. LID auto-switch still respects this list (without * wildcard).
voice_advanced.voice.profilesByLanguage	{}	STT and TTS model per language, used after a language switch. JSON object keyed by ISO code with optional stt, tts (Sherpa catalog ids), sttVendor, ttsVendor, and voice (cloud voice id). The dashboard shows it as a table. Example: {"de":{"stt":"de","tts":"de-thorsten-high"},"es":{"ttsVendor":"elevenlabs","voice":"your-voice-id"}}. Unknown Sherpa ids fail at save.
voice_advanced.events.mode	both	How speech events reach your agent: callback, async iterator stream, or both. Cloud runners use both by default. Change only if your agent bundle expects a specific delivery mode.

Example scenarios

Noisy room — reduce false speech detection

Raise vad.threshold (e.g. 0.2–0.25).
Increase vad.minSpeechDurationMs to ignore brief noise bursts.
Keep semantic barge-in enabled.

Faster replies — shorter pauses

Lower vad.minSilenceDurationMs (e.g. 800–1000). The gate hold already defaults to 400 ms.
Watch for split sentences and truncated last words — raise the values if callers get cut off or finals clip endings.
See How the end of a turn is decided for what each wait does.

Speaker echo interrupts agent TTS

Keep vad.bargeIn.requireSttPartial true (default).
Optionally raise vad.bargeIn.agentPlaybackGuardMs (e.g. 200–500) after trying headphones or lower speaker volume.

Broadcast agent — no user interrupt

Set vad.bargeIn.enabled to false.

Self-hosted agents

If you run @node-webrtc-rust/sdk locally, pass the same fields on VoiceAgentConfig. See the open-source VAD and barge-in guide for deep tuning timelines and event order.

Library default snapshot: gateStt=true, speechPadMs=1000, requireSttPartial=true.