Latency

Latency is the time between an event and the moment a system can act on it. Every real sensor has some. A camera exposes a frame, the image is sent and processed, and only then does a controller see it. A human eye and brain are no different: what you see is already slightly in the past.

For a person, the shortest delay is a simple reaction: one known signal, one known response, like a sprinter’s start. It takes roughly 150–200 ms even for elite athletes. A choice reaction, where you must first decide which of several responses is right, takes longer.

In a simulation there is no delay unless you add one. A policy trained without it can learn to react at the last instant to things no real body could react to in time, and that skill would not survive on a real robot. So the observation is held back by a fixed time on purpose. If the delay is $d$ and a punch’s first visible movement comes $t_{\text{lead}}$ before it lands, then the time left to block is

$$t_{\text{left}} = t_{\text{lead}} - d - t_{\text{block}}$$

where $t_{\text{block}}$ is how long the block itself takes to get into place. When $t_{\text{left}}$ is below zero, reacting to what you see can’t work: you need to guess earlier or be out of range.

In this project

In the blocking post, Tony chose a vision delay of 100 ms, about a fast professional’s. The opponent’s keypoints reach the fighter 100 ms after they happen. The jab’s first visible movement comes only 0.10 s before it lands, so after the delay there is no time left at all to react to it. The hooks give 0.23–0.24 s of warning, enough for most of a block.