Fix Whisper Hallucinations on Silence
Why Whisper types "Thank you" or "Thanks for watching" on silence, and the fix: VAD, condition_on_previous_text off, and no-speech guards in faster-whisper.
Whisper hallucinates on silence because its decoder is built to keep producing text, so when it is given a near-empty audio window it fills the gap with the most statistically likely caption-style phrase, such as "Thank you" or "Thanks for watching". The fix is to stop silence reaching the model: run voice activity detection (VAD) first, and turn off previous-text conditioning so one bad line cannot repeat forty times. Whisper also ships its own no-speech and log-probability guards, which help but do not replace a VAD gate.
Here is why it happens, the settings that fix it in faster-whisper and similar runtimes, and how to avoid the problem in daily dictation.
What the hallucination looks like
You press record, say nothing for a few seconds, and the transcript contains a sentence you never spoke. The usual suspects are short, polite, caption-style phrases. In a test of non-speech audio reported by Dograh, "Thank you" made up 24.76% of outputs and "Thanks for watching" another 10.32%, with "Subtitles by the Amara.org community" appearing in a small fraction. That is one third-party test, so read it as indicative, not as a benchmark.
The second failure mode is a loop. The model repeats the last phrase it recognised, over and over, until the segment ends.
Why Whisper does it
Two causes show up consistently in the maintainers' discussion threads and in practitioner write-ups:
- No stop signal. Near-silent audio gives the encoder almost no useful signal, but the decoder still has to emit tokens. It predicts the most probable continuation, and given training data heavy on subtitled video, that is often a caption sign-off.
- Previous-text conditioning. With
condition_on_previous_texton, each window is decoded with the previous output as context. If the current window is unintelligible, the model can copy its earlier text instead of emitting nothing, and one error becomes a loop. In the openai/whisper discussion on this, turning the parameter off makes the model emit blank output in that situation instead.
This is not a rare edge case in research either. The "Careless Whisper" study (Koenecke et al., FAccT 2024) found that about 1% of audio transcriptions contained entire hallucinated phrases or sentences that did not exist in the audio, and that 38% of those hallucinations included explicit harms. The authors tied hallucinations to longer non-vocal durations, which disproportionately affected speakers with aphasia. The study tested Whisper as it stood in 2023, and the lead author has said OpenAI has since improved the model, so treat the 1% as a historical baseline rather than today's rate.
The fix stack
Apply these in order. The first two do most of the work.
| Fix | What it does | Trade-off |
|---|---|---|
| VAD before Whisper | Sends only detected speech to the model | Slightly clipped word edges if set too aggressively |
condition_on_previous_text=False | Stops one bad line seeding the next window | Can slightly reduce consistency of punctuation and style across long audio |
no_speech_threshold (default 0.6) | Marks low-confidence segments as silence | Too high skips quiet real speech |
logprob_threshold (default -1.0) | Rejects low-probability decodes | Too tight drops genuine hard-to-hear words |
hallucination_silence_threshold | Skips long silences near a suspected hallucination | Needs word timestamps enabled |
| Phrase and loop filter | Strips known sign-offs and repeated lines after decoding | A safety net only; never a first line of defence |
Step 1: gate silence with VAD
faster-whisper integrates the Silero VAD model. Its README describes the default as conservative, removing silence longer than 2 seconds, and shows how to tune it through vad_parameters.
from faster_whisper import WhisperModel
model = WhisperModel("large-v3-turbo", device="auto", compute_type="int8")
segments, info = model.transcribe(
"meeting.wav",
vad_filter=True,
vad_parameters=dict(min_silence_duration_ms=500),
condition_on_previous_text=False,
)
for segment in segments:
print(f"[{segment.start:.1f}s] {segment.text}")
A shorter min_silence_duration_ms cuts more silence but risks trimming natural pauses. Start at 500 ms and listen to the result on your own recordings.
Step 2: break the feedback loop
condition_on_previous_text=False is the single-line change that stops a repeated phrase from snowballing. The faster-whisper README itself uses it in its distil-large-v3 example. The cost is that long-form output may lose some cross-window consistency, so for short dictation clips it is nearly free.
Step 3: tighten the decoder guards
If VAD still lets near-silence through, a faster-whisper issue thread suggests a volume gate alongside VAD, since very quiet noise is not always removed. Adjust no_speech_threshold and logprob_threshold carefully, and test on quiet-but-real speech, because an aggressive setting will delete genuine words.
Step 4: post-filter as a last resort
A filter that deletes known sign-off phrases and collapses repeated lines catches what remains. Dograh reports that combining Silero VAD, a de-looping filter, and a phrase list cut word error rate on a hard non-overlapping test set from 104.8% to 6.5%. That figure is the vendor's own and comes from adversarial audio, so do not expect those numbers on normal speech, but it shows how much of the damage comes from silence handling.
Never rely on a phrase filter alone. It hides the symptom while the model still wastes time decoding silence, and it can delete a real "thank you" you actually said. Gate silence first, filter second.
Why this matters more for dictation
In batch transcription, a stray "Thank you" is an annoyance you fix in review. In system-wide dictation it lands in your email, your terminal, or a chat box the moment you stop speaking. Two habits reduce the risk in any tool:
- Stop recording when you stop talking. Push-to-talk or an auto-stop mode keeps long silent tails out of the audio entirely.
- Keep the mic clean. Fan noise and open-mic hiss give the decoder just enough fake signal to latch onto. A closer microphone helps more than any threshold.
Local Whisper has an advantage here: you control the whole pipeline and can see exactly what the model receives. Nothing is uploaded, so there is no vendor-side filtering to trust or distrust. For a wider look at where Whisper holds up and where it does not, see how accurate Whisper is for meetings, and for setting it up yourself, run Whisper locally for dictation. If you are choosing between engines, Whisper large-v3-turbo vs Parakeet covers the trade-offs.
The short version
Whisper invents "Thank you" and "Thanks for watching" on silence because its decoder always tries to output something and its training data is full of subtitle sign-offs. Gate silence with VAD, set condition_on_previous_text=False, tune the no-speech guards carefully, and use a phrase filter only as a backstop. Typilot runs Whisper on your device with push-to-talk and VAD auto-stop modes, see dictation, so recording ends when you stop speaking and nothing leaves your machine.
Try Typilot free for three days at /download. The architecture proof is at /security.
Common questions.
Why does Whisper say "Thank you" or "Thanks for watching" when nobody is talking?+
Whisper's decoder is built to keep emitting tokens. On near-silent audio the encoder has almost no signal, so the decoder predicts the most probable text, and caption-style sign-offs are common in its training data. A third-party test reported by Dograh found "Thank you" in 24.76% of outputs on non-speech audio.
How do I stop Whisper hallucinating on silence?+
Run voice activity detection before Whisper so only speech reaches the model, and set condition_on_previous_text to False so one bad line cannot seed the next window. In faster-whisper that is vad_filter=True plus condition_on_previous_text=False. Then tune no_speech_threshold (default 0.6) carefully and use a phrase filter only as a backstop.
How common are Whisper hallucinations?+
The Careless Whisper study (Koenecke et al., FAccT 2024) found about 1% of audio transcriptions contained entire hallucinated phrases or sentences, and 38% of those hallucinations included explicit harms. It tested Whisper as of 2023, and the lead author has said OpenAI has since improved the model, so the figure is a historical baseline.
Does running Whisper locally avoid hallucinations?+
Local Whisper is the same model, so it can hallucinate the same way. The difference is control: you choose the VAD, the decoding settings, and what the model receives, and nothing is uploaded. Push-to-talk or VAD auto-stop recording also keeps long silent tails out of the audio.