Stream Latency Is Not Lip-Sync Drift

Two complaints sound almost identical and have nothing to do with each other. "Viewers see it late" is latency. "The lips do not match the voice" is drift. They are fixed in different places, by different people, and the first useful thing you can do is refuse to treat them as one problem.

Telling them apart in ten seconds

The distinction that matters most: latency is a delay you can budget for, and sync is a fault. Ten seconds of latency is a design decision. Two hundred milliseconds of lip-sync error is broken, at any latency.

What makes up the delay

Latency is not one number produced by one thing. It is a sum, and each term belongs to someone different. Knowing which term is which tells you immediately whether you can do anything about it.

Your encoder. It has to collect enough frames to compress them against each other, then hand out the result. Small, and mostly fixed by your encoder's own buffer settings.

The path to us. Physical distance plus whatever protection you asked for. If you are using SRT, the URL carries a latency budget of at least 500 ms — you have explicitly bought that much delay in exchange for surviving packet loss, and that is the trade working as intended, not a fault.

Our encoding and packaging. The stream is re-encoded into the quality ladder and cut into segments. dcast uses 4-second segments with a 2-second keyframe interval, so a segment cannot be published until 4 seconds of it exist. This term is the largest one in the chain for segmented delivery, and it is arithmetic rather than inefficiency.

Delivery to the viewer. Getting the segments to wherever the viewer is.

The player's buffer. Before it shows anything, the player collects segments so that a slow moment on the viewer's connection does not become a visible stall. This is the term most people do not know exists, and it is often the biggest one after packaging. A player holding three 4-second segments is twelve seconds behind live before it has drawn a single frame — not because anything is slow, but because it decided to be safe.

What you can and cannot reduce

Add the terms up and it becomes obvious where the room is. The physics — distance, the speed of the network — you cannot change. Your encoder's buffer is small enough that tuning it is rarely worth the risk. The SRT latency budget you can lower, but you lower your tolerance for packet loss by exactly as much, and if you were using SRT you presumably had a reason.

The two terms with real room in them are the segment duration and the player's buffer, and they are linked: the player buffers in units of segments, so the segment length multiplies through. On dcast the player side is exposed as a latency mode on the stream — normal, low or ultra — and it is worth being precise about what that setting is: it changes how much the player buffers, and it does not reach the encoder at all. The encoding grid is the same for every stream on the platform.

That is the honest shape of the trade. A shorter buffer means the viewer is closer to live and has less protection against their own connection hiccuping; a longer buffer means the opposite. There is no setting that gives both, and anyone who tells you a protocol eliminates the trade has moved it somewhere else rather than removed it.

Decide how live your stream needs to be

Most streams do not need low latency, and the ones that do usually need it for one specific reason that is worth naming before you start trading away robustness.

If your viewers interact with you live — you read chat aloud, you take questions, you run an auction — then latency is the whole product, and a viewer who is twenty seconds behind is participating in a different conversation from yours.

If your viewers might see the event twice — a sports fixture where the crowd next door reacts first, or anything where a phone notification can beat your stream — latency is a spoiler problem.

If neither applies, and most broadcasts are neither, then a comfortable buffer is the better engineering choice, because the failure it prevents (a stall in the middle of the talk) is more visible to a viewer than being ten seconds behind an event they have no other view of.

The latency you add to yourself

Before blaming delivery, it is worth counting the delay that exists inside your own room, because it is often larger than people expect and it is the part you can actually remove.

Every device the signal passes through costs something. A camera, a converter, a switcher, a scaler, a second switcher feeding a projector — each one holds the picture for a moment while it does its work. A chain of four devices is four delays, and none of them is documented anywhere you will find easily.

Wireless links cost more than wired ones. A wireless video transmitter has to packetise, protect and reassemble; that protection is exactly the same trade as an SRT latency budget, made by a device instead of by you.

A relay costs a full trip. If your signal goes from the venue to an office and from there to the platform, you are paying for both legs plus whatever the middle box does. It is common to discover that the "streaming latency" someone is complaining about is mostly a hop that exists for historical reasons.

The way to find it is to put a clock in front of the camera and photograph the clock together with each screen along the chain: the camera's own monitor, the switcher's programme output, the encoder preview. Each step tells you what that stage cost. Very often one device turns out to be responsible for most of it, and removing that device from the path is faster and cheaper than any settings change downstream.

Lip-sync is a different fault entirely

Now the other complaint. If the voice and the lips disagree, changing your latency settings will not help, because the audio and the video are travelling together — the whole packaged stream is late as one thing. A sync error means the two were already misaligned when they arrived, or something separated them.

Almost always, this happens before the encoder. Sound and picture reach your encoder through different equipment, and different equipment takes different amounts of time.

Video is usually the slow one. A camera processes and scales, a capture device converts, a switcher composites. Each step costs a little. Audio, by contrast, often arrives by a much shorter route — a microphone straight into an interface.

So the usual fix is to delay the audio, not to advance the video, because you cannot advance anything: the picture is already as early as it will ever be. Nearly every switcher and audio interface offers an audio delay for precisely this reason.

Measure it before you set it. Clap once in front of the camera and the microphone at the same time, record the programme output, and look at where the clap lands on each track. That gives you a number instead of a guess, and the number is usually stable for a given equipment chain, so you set it once.

When the offset grows during the broadcast

A fixed offset is annoying and easy. An offset that starts small and is a second wide an hour later is a different fault, and delay settings will not hold it.

That is a clock problem. Your audio device and your video device each count time with their own oscillator, and if they disagree by even a tiny fraction, the gap accumulates for as long as you are on air. A hundredth of a percent is a third of a second per hour.

The fix is to make them share a clock rather than to correct the symptom. Where your equipment supports it, drive audio and video from one reference. Where it does not — the common case with consumer gear — prefer paths that keep them together: audio embedded into the video signal over HDMI or SDI, or captured by the same device that captures the video, rather than a separate audio interface with its own crystal.

If neither is available, the practical mitigation is to keep individual broadcasts short enough that the accumulated error stays under the threshold anyone notices, and to re-align between segments.

A diagnosis order that works

Check your own programme output first, on your own machine, before doing anything else. If the sync is already wrong there, nothing downstream did it and no platform setting will repair it.

If the local output is correct, check a recording of the delivered stream. Sync that is right locally and wrong downstream is genuinely unusual and worth reporting, because it is not a settings problem.

Only once sync is confirmed correct should you look at latency at all. Correcting a delay while the audio is out of step just gives you a faster wrong stream.

And measure the latency the way a viewer experiences it, not from your encoder's statistics: put a clock on screen, watch the stream on an unrelated device, and photograph both. That number includes the player buffer, which is the term your encoder cannot see and which is frequently the largest one.