feat(stt): pre-upload silence trim for cloud providers

Local faster-whisper gets Silero VAD (bf8004e3a) so silence never
reaches the model. Cloud providers got no such protection: the raw
file uploads untouched, so every second of silence in a voice note is
paid for twice — upload time and per-audio-minute billing — and cloud
Whisper hallucinates junk tokens on silent stretches exactly like
local Whisper did before the VAD hardening. A 13s voice note with two
long pauses is billed as 13s of audio to transcribe ~6s of speech.

Close the gap client-side: before uploading to a built-in cloud
provider (groq/openai/mistral/xai/elevenlabs/deepinfra), collapse long
pauses with ffmpeg's silenceremove filter, keeping
stt.cloud_trim_keep_ms (default 300) of every pause so word boundaries
and natural pacing survive. Uses ffmpeg, already a dependency of this
exact path via _transcode_audio_for_stt — no new dependency.

The trim is strictly best-effort — ALL of these upload the original
untouched, transcription never fails because of the trim:
  - stt.cloud_trim_silence: false
  - ffmpeg/ffprobe missing, trim failure, or timeout
  - trimmed result ~empty (mostly-silence clip: the provider, not a
    client-side dB heuristic, decides whether it contains speech)
  - trim saves <10% (re-encoding for nothing)

Command-type and plugin providers are deliberately NOT trimmed: they
may wrap local CLIs that want the original bytes or run their own VAD.

E2E (real ffmpeg + faster-whisper): 13.2s voice note with 7s pause ->
6.2s upload (-53%); transcript of trimmed audio matches the original
on both utterances. Dense-speech and all-silence WAVs correctly fall
back to the original. 22 unit+E2E tests; STT/voice suite failures
identical to upstream/main baseline (all pre-existing).
This commit is contained in:
kshitij
2026-08-03 16:14:30 +05:30
committed by kshitij
parent 83a1ca6862
commit a683ef95d2
5 changed files with 493 additions and 0 deletions
+7
View File
@@ -1193,6 +1193,13 @@ platform_toolsets:
stt:
enabled: true
# provider: "local" # auto-detected if omitted
# --- Cloud pre-upload silence trim (groq/openai/mistral/xai/elevenlabs/deepinfra) ---
# Local whisper gets Silero VAD; cloud endpoints otherwise receive raw audio —
# silence inflates upload time, per-audio-minute billing, and hallucination risk.
# Collapses pauses with ffmpeg client-side; on any failure the original uploads untouched.
# cloud_trim_silence: true # set false to always upload the original audio
# cloud_trim_threshold_db: -40 # audio quieter than this counts as silence
# cloud_trim_keep_ms: 300 # how much of each pause survives (keeps natural pacing)
local:
model: "base" # tiny | base | small | medium | large-v3 | turbo
# language: "" # auto-detect; set to "en", "es", "fr", etc. to force