How voice filters work in a dubbing game

Every voice changer you have used works the same way underneath: it resamples. Play the recording faster and you go up, slower and you go down. It is one line of code, it is what a tape machine does, and it is the reason the chipmunk voice exists at all.

It is also the one thing a dubbing game cannot do, and working out why is most of what makes voice filters in this genre interesting.

Rate changes pitch and duration together

In a dub, your take is not played on its own. It is scheduled against a picture — a character opens their mouth at 4.2 seconds and stops at 6.1, and your line goes in that gap. Nothing stretches it to fit and nothing cuts it short.

So the moment you change playback rate you have changed how long the line lasts. A chipmunk at 1.9× finishes halfway through, with the character's mouth still moving. A giant at 0.55× is still talking seconds after the scene has cut away. Both are the exact desynchronisation every other part of a dubbing game exists to prevent — and the funnier the filter, the more extreme the rate, and the worse the drift.

This is why a voice changer built for chat or streaming often sounds wrong the first time somebody points it at a dub. Nothing is broken. Chat has no picture to stay in sync with, so nobody had to solve this.

Formants, and why they carry more than pitch does

The fix is to stop moving pitch at all and move the resonances instead. Your vocal tract is a tube, and a tube has frequencies it rings at — formants. A short tract rings high, a long one rings low, and the length of somebody's tract is most of what makes a voice read as small or large. Arguably more than pitch is.

Moving where a voice resonates is pure EQ. In PartyDub every filter is a chain of peaking filters at roughly 240 Hz for chest resonance, 1400 Hz where the vowels live, and 3000 Hz for the brightness that reads as a lighter voice. Lift the top two and cut the bottom one and the speaker gets smaller; do the reverse and they get bigger. The recording is never resampled, so it lasts exactly as long as it was performed, to the sample.

The trade is honest and worth stating: a formant-shifted chipmunk is a small person, not a literal chipmunk. It is less extreme than resampling. It also stays on the picture, which in a game where the joke is a voice against a face matters more than the last twenty per cent of the effect.

The ones that are not about size

Not every filter is a formant shift. Several are older tricks that happen to be free in a browser:

• —Robot is ring modulation — the signal multiplied by a 55 Hz sine wave. Web Audio has no multiply node, but a gain node's gain parameter is a multiplier, so driving it with an oscillator is exactly ring modulation.
• —Angry wasp is the same trick at 260 Hz, squeezed into a narrow band. Fast modulation into a bandpass is pure buzz.
• —Phone call is the 300–3400 Hz voice band and nothing else. That band is what makes a telephone sound like a telephone; it is a fact about the phone system, not about the voice.
• —In a cave is a feedback delay at 220 ms with the top rolled off, which is roughly what a real cave does to you.
• —Demon is a big downward formant shift with waveshaper distortion on top.

All of it runs in the browser at the moment the room plays a dub back. Nothing is uploaded, nothing is rendered on a server, and the effect costs the same whether one person hears it or ten do.

The design problem underneath the audio one

Getting eighteen filters to sound good turned out to be the easy half. The hard half was that for a long time they were a dropdown: you opened a menu before recording, picked Demon, and got Demon.

That is a preference, and a preference is a lonely thing in a party game. Nobody else in the room has a stake in it, nobody is surprised by it, and once you have found the one you like you stop opening the menu. A funny voice you selected for yourself is a costume you are wearing alone.

So filters in PartyDub are no longer chosen. They are thrown. There is a shelf of eighteen potions during a round; you pick a bottle, pick a player, and everybody watches it arc across the screen and burst on that person's card. For the rest of the round their voice comes out through that filter — in the booth while they are still recording, and at the reveal when the room hears it.

The person being hit gets no say in it, which is the entire point and the one thing a dropdown can never do.

Two rules that keep it from being annoying

1. A potion lasts one round. Next round you are clean again unless somebody spends another one on you. Being hit in round one and stuck as a wasp for an hour is a punishment; being hit for a round is a bit.
2. You are told immediately. Playing your take back in the booth, you hear what you actually sound like. Hiding it to save the surprise was tempting and would have been worse — a performer who does not know they sound like a demon cannot play up to it, and a line read straight because you had no idea is a joke that lands on nobody.

One throw each per round, so in a room where several people are holding potions a round can have four bottles crossing at once. Being hit needs no account and no plan: the joke does not work if half the room cannot be a target.

What it costs

A signed-in free account gets one potion a month. Supporters get fifty. One rather than none because a number on a pricing page means nothing until you have thrown one; one rather than five because scarcity is the whole reason a potion is worth anything. Guests can be hit but cannot throw, for the unglamorous reason that a monthly allowance needs an account to hang off.

Either way the audio is identical. Every filter is a few Web Audio nodes running on the machine that plays the dub, which costs us nothing and costs you nothing — the allowance is about keeping a thrown potion an event rather than weather.