How audio noise reduction works, and why voices go watery
Every recording made outside a studio has a floor: fan, air conditioning, hum, hiss, the rumble of a room. Noise reduction lifts the voice out of that floor, and the standard technique — the one in Audacity, most podcast tools and this site's noise reducer — is spectral gating. It is worth understanding because it explains both what it cleans beautifully and the "underwater" sound it produces when pushed.
What counts as noise to an algorithm
To the software, noise is whatever is steady: present in the same way throughout the recording, with a consistent spectrum. Hiss, hum, fans, refrigerators and room tone all qualify. A dog barking, traffic, a door, a keyboard, a second voice — sounds that come and go — are not noise in this sense, and a gate can't touch them without also cutting the speech they overlap. That single distinction predicts most successes and failures.
Seeing sound as a spectrogram
The processor cuts the audio into short overlapping windows (about 40 milliseconds) and takes a Fourier transform of each, giving the energy at every frequency in that slice — a column of a spectrogram. Speech shows as moving bands of energy (the harmonics of the voice and the formants that shape vowels); steady noise shows as a flat, constant background across every column. Separating them is now a comparison, frequency by frequency, time slice by time slice.
Spectral gating, step by step
- Learn the noise print. From a stretch of "silence" — selected by hand in Audacity, or found automatically as the quietest 10% of windows — the average energy at each frequency is measured. This is the noise's fingerprint.
- Set a threshold per frequency. Typically a few times the noise level; the "sensitivity" or "strength" control.
- Gate each bin. For every window and frequency, if the energy sits near the noise level, turn it down (by up to the chosen reduction, say 20 dB); if it rises clearly above, leave it alone. Speech harmonics pass; the hiss between them is attenuated.
- Smooth. Gates that snap open and shut per window sound terrible, so the gains are smoothed over time and across neighbouring frequencies.
- Resynthesise. Inverse transform each window and overlap-add them back into a waveform.
Everything above is arithmetic on your device; a three-minute file takes a few seconds in a browser.
Musical noise and the watery voice
Push the threshold up and the gates start cutting into the quieter parts of speech — breathy consonants, the tails of words — while random bins of noise near the threshold flicker open and shut. The flickering produces short chirps called musical noise; the cut consonants produce the hollow, underwater voice every over-processed podcast has. The cure is restraint: moderate strength, a cap on the reduction (12–20 dB rather than 40), and accepting a little residual hiss, which listeners ignore, over a voice that sounds processed, which they don't. Newer AI denoisers model speech rather than noise and avoid this artefact, at the cost of a large model and, usually, an upload.
Getting a cleaner recording in the first place
Every decibel of noise you don't record is one you needn't remove. Get the microphone close (halving the distance gains 6 dB of voice over room noise), turn off fans and the fridge, record in a room with soft furnishings, and leave two seconds of silence at the start so any tool can learn the noise print cleanly. Then normalise the level, trim the pauses and encode to MP3. The voice recorder shows a level meter to help you set it up.
Sources and further reading
The claims in this guide rest on these references, which were checked when the guide was last updated. Spotted an error? The contact page says how to report it.