I've burned more hours chasing "bad model accuracy" than I care to admit, and half the time the model was fine. The audio was the problem: phone recordings at 8 kHz mono, stereo podcasts with music beds, Zoom exports with clipping, or WAV files that somehow arrived as 48 kHz with a DC offset. Once I started treating audio preprocessing for transcription as a first-class step with FFmpeg, word error rates dropped before I touched a single API parameter.
In our complete guide to speech-to-text for AI agents, we cover the full voice pipeline. This guide focuses on the audio prep layer: the FFmpeg settings I use before sending files to a private speech-to-text API.
Why preprocessing beats prompt tuning
Most transcription APIs expect clean, speech-dominant audio in a predictable format. When you skip normalization, you're asking the model to compensate for sample-rate mismatches, channel bleed, and level swings that a two-line FFmpeg command would remove.
I've A/B tested the same 200-clip evaluation set with and without a fixed preprocess step. Preprocessing alone cut relative WER by roughly 8-15% on phone and meeting audio, enough that we stopped blaming the STT vendor for problems we introduced upstream. If you're feeding transcripts into an LLM, cleaner audio also means fewer hallucinated words that waste tokens later; see how clean transcripts cut LLM costs.
| Problem in source audio | Symptom in transcript | FFmpeg fix |
|---|---|---|
| 8 kHz telephony | Muffled consonants, name errors | Resample to 16 kHz with high-quality filter |
| Stereo with music bed | Lyrics bleed into speech text | Downmix to mono; highpass filter |
| Clipping / hot levels | Garbled peaks, false words | Loudnorm + limiter before encode |
| Long silence / hold music | Wasted minutes and cost | Silence detect / trim before upload |
Bottom line: Fix the waveform before you tune the model. It's cheaper and more repeatable than vendor-hopping.
The FFmpeg baseline I use on every file
My default target for English speech APIs is 16 kHz mono PCM (or high-bitrate mono Opus/MP3). That matches what most Whisper-class and commercial STT stacks were trained on. Here's the command I run as a gate before any batch upload:
ffmpeg -y -i input.wav \
-af "highpass=f=80,loudnorm=I=-16:TP=-1.5:LRA=11" \
-ac 1 -ar 16000 -c:a pcm_s16le \
prepared.wav
What each piece does:
highpass=f=80removes rumble and HVAC noise that models sometimes treat as speech energyloudnormbrings levels to a consistent loudness target so quiet speakers don't disappear-ac 1 -ar 16000forces mono 16 kHz, the sweet spot for speech, not musicpcm_s16lekeeps a lossless intermediate for debugging; I re-encode to Opus only for upload size
If the source is already compressed (M4A, MP3), I still decode, filter, then re-encode rather than filter in place. I've seen double-compressed MP3s introduce artifacts that look like "model hallucinations" in the transcript. For a fuller batch workflow, pair this with our batch transcription best practices.
When you're ready to send the file, Privocio's API docs accept standard audio formats. Just don't skip the sample-rate step on telephony sources.
Noise, loudness, and when to stop processing
Aggressive denoising feels productive and often hurts. Spectral subtraction and heavy RNN denoisers can smear fricatives so "fifty" becomes "city." I only add afftdn or similar when SNR is genuinely bad (call centers with fan noise, outdoor recordings). Start mild:
ffmpeg -y -i input.wav \
-af "highpass=f=80,afftdn=nr=12:nf=-25,loudnorm=I=-16:TP=-1.5:LRA=11" \
-ac 1 -ar 16000 prepared.wav
Run a 20-file holdout before and after. If WER doesn't move, drop the denoiser. Loudness normalization almost always helps; exotic filters rarely do.
For multi-speaker meetings, don't "fix" by collapsing everything into a crushed mono track with music ducking. Keep speech intact and let diarization (if you need it) run on clean audio. Music beds and intro stingers should be trimmed with silenceremove or a simple start/end crop. Uploading 45 seconds of theme music is pure cost with zero transcript value. Our pricing is fixed-rate, but wasted minutes still slow your pipeline and burn LLM tokens on junk text.
Batch pipeline pattern for agent workloads
In production agent systems, preprocessing sits between ingestion and the STT call. I keep it in a worker that writes a deterministic prepared.* artifact so retries never re-run FFmpeg on the original with different flags.
Normalize once and store the prepared file so transcription and reprocessing always hit that path. Fail closed on corrupt media: if FFmpeg exits non-zero, quarantine the job instead of uploading garbage. Log ffprobe sample rate, duration, and channels into your job record so you can debug WER spikes later. Prefer batch over live when prep is heavy; real-time streams can't always wait for loudnorm, but recorded prompts should be cleaned first (see real-time vs batch).
I wire this into Privocio the same way I'd call any Whisper-compatible endpoint: preprocess locally or in your VPC, then POST the prepared file. If you need audio to stay on your infrastructure, use self-hosted or private cloud options from the features overview and keep FFmpeg in the same network boundary. For a quick sanity check on a single file, the browser transcribe tool is fine. Just preprocess first if the source is a phone recording.
Frequently Asked Questions
Do I need 16 kHz, or will 8 kHz telephony work as-is?
Most modern STT models accept 8 kHz, but I've consistently gotten better name and digit accuracy after upsampling to 16 kHz with a high-quality resampler. Upsampling doesn't invent missing bandwidth, but it matches the model's expected input grid and reduces edge-case bugs in some clients.
Should I use MP3 or WAV for API uploads?
Use WAV/FLAC/PCM for the preprocess intermediate. For upload, a high-bitrate mono Opus or MP3 is usually fine and smaller. Avoid re-encoding already-lossy files multiple times. That is where quality dies.
Can FFmpeg replace a dedicated noise-reduction product?
For most agent and call-center workloads, yes. Highpass plus mild afftdn plus loudnorm covers the common cases. Dedicated tools help for extreme forensics audio; they are overkill for everyday STT pipelines.
Does preprocessing help with Privocio's Agent output mode?
Indirectly. Cleaner audio produces fewer garbage tokens in the raw transcript, so Agent/Clean modes have less junk to strip. Prep the audio first, then pick the output mode that fits your LLM.
Conclusion: Fix First, Then Transcribe
I've stopped treating transcription accuracy as purely a model problem. A short, boring FFmpeg baseline (mono 16 kHz, highpass, loudnorm) fixed more "bad STT" tickets than any prompt tweak. If you're building voice agents, preprocess in your worker, store the prepared file, then call your STT API. Start free on the transcribe tool or check pricing for fixed-rate plans once volume grows. For the wider agent stack, return to our speech-to-text for AI agents pillar.
Image Credits:
Cover image sourced from Unsplash (Unsplash License).