Whisper for podcasts — transcribe every episode locally, for free

Why Whisper is the podcast default in 2026

Three reasons, and none of them involve an API key. First, accuracy: on clean English studio audio modern Whisper-class engines land in the 3–6% word-error-rate band — the same numbers the paid cloud services inherit, because most of them run Whisper or a fine-tune of it under the hood. Second, it runs on your hardware: the model file sits on disk, inference happens on your CPU or GPU, and nothing uploads. For interview shows with source-protection concerns, that is not a convenience — it is the whole point (no vendor ever receives the audio, so there is nothing to subpoena, breach, or train on). Third, it is genuinely free software with no minute caps: a ten-year back catalog of 300 episodes costs the same as one episode — zero.

The honest caveat: real podcast audio is harder than benchmarks. On a real two-person show — remote guests, one accent, crosstalk, brand names — expect 8–15% WER and a short cleanup pass, not the marketing 98%. The good news is that the two highest-leverage fixes are free and take minutes: fix your input files (next section) and feed the model a custom vocabulary list for guest names, product names, and jargon. Every engine mishears the same 30–50 words episode after episode unless you teach it once.

Start from per-track source files, not the mixed feed

The single biggest quality decision happens before Whisper ever runs: which audio file you feed it. Remote recording studios keep the goods — Riverside, Zencastr, and StreamYard all record each participant as a separate track. Use those.

Two mechanical notes. If you transcribe each track separately, you already know who is speaking — label the files and merge, and skip diarization entirely (on one-voice-per-track input it is nearly error-free by construction). And mind the bitrate chain: transcoding a 320 kbps master down to 128 kbps before transcription costs accuracy you never get back — transcribe from the highest-quality file you have, then compress for publishing.

The full step-by-step — from getting the episode audio to publishing the transcript page — is in our podcast transcription workflow guide.

Model size vs accuracy: which Whisper to run

One decision, three sensible answers. The 2026 default is large-v3-turbo: released September 2024 with the decoder pruned from 32 layers to 4 (809M parameters), it runs roughly 5× faster than full large-v3 with near-identical accuracy on well-resourced languages like English — and it made the old medium-model compromise obsolete, beating medium on both speed and accuracy at nearly the same footprint.

Benchmarks for context: NVIDIA's Parakeet TDT 0.6B v3 posts a 6.34% average WER vs Whisper large-v3's 7.44% on published English benchmarks, so if you are choosing engines rather than models, the Whisper label is not automatically the accuracy crown. Whisper's advantages are the 99-language coverage and the ecosystem (every tool speaks it).

The hallucination problem: intro music, silence, and "Thanks for watching"

Every podcast has the exact audio that trips Whisper: a produced jingle, room tone between takes, thirty seconds of dead air at the tail. Whisper was trained on weakly supervised audio paired with subtitle files — segments where the text does not match the audio — so it learned real associations between non-speech sound and fluent text. Feed it your intro music and it may return "Thanks for watching," a subtitle credit, or the same phrase looping until the segment ends. The outputs look plausible — well-formed timestamps, confident text — which is exactly what makes them dangerous in an automated pipeline: nothing marks them as fabricated.

The fix that actually works is VAD — voice activity detection — so non-speech audio never reaches the model. Whisper.cpp ships Silero VAD support:

whisper-cli -m models/ggml-large-v3-turbo.bin \
  --vad -vm models/ggml-silero-v5.1.2.bin \
  -f episode.wav -osrt

Two tuning flags matter most. --vad-speech-pad-ms pads each detected speech segment — set it too low and word onsets get clipped (you trade hallucination for truncation). --vad-min-silence-duration-ms decides how long a pause must be before a segment is cut — too low and sentences fragment, costing accuracy. And note the classic gotcha: --vad needs the VAD model passed via -vm — omit it and the flag appears to do nothing.

If loops persist, escalate: disable cross-segment conditioning (-mc 0), consider --no-timestamps for pure-text runs (timestamp computation is implicated in repetition loops), trim music beds manually, and post-filter the specific boilerplate phrases your audio produces. Confidence-threshold flags alone disappoint: hallucinated boilerplate is high-confidence output — the filter is designed to catch low-confidence text, so raising thresholds starts discarding quiet real speech before it reliably discards confident invention.

Batch workflows: a back catalog on autopilot

The transcription itself is fast; the reason to think in pipelines is everything around it. Published local-pipeline timings for a 90-minute episode: ~5.5 minutes on an RTX 4070, ~8.5 on an RTX 3060 — while CPU-only machines took ~58 minutes for the same file. If you are GPU-less, batch overnight instead of waiting; if you are GPU-equipped, a full back catalog of 40 episodes is an afternoon, not a month.

A practical local stack decouples the stages so each is replaceable:

That last step is where the SEO value compounds: the same transcript becomes Google-indexable episode text, chapter markers, show notes, and episode schema. Transcribe once, publish everywhere.

Turnkey option: Whisper-class transcription without the command line

The DIY stack above is the free-and-flexible route, but it is a dev project: you maintain the runtime, the VAD model, the diarization plumbing, and every model update. VocalFuse ships the same architecture as a Windows app — a ~57 MB Whisper-class engine that runs entirely on your machine. Drop the episode in, get a timestamped transcript with speaker labels, export TXT/SRT/VTT. Nothing uploads, ever.

Because the engine runs on your PC, there is no per-minute meter anywhere in the product — the free tier covers real work, Basic is $5/mo flat dictation, and Pro $10/mo adds AI note taking, speaker labels, and summaries. Local processing means your raw interview audio never touches a vendor's cloud — the same guarantee the DIY stack gives you, without the pipeline maintenance.

Related reading

Whisper for podcasts — FAQ