Phase variable

A recorded or animated movement is a list of poses over time. To follow it, a controller needs to know where in the movement it is. A phase variable $\phi$ answers that: 0 at the start, 1 at the end. The pose to aim for is looked up from the phase, not from the wall clock.

The phase usually advances at a steady rate:

$$\phi_{t+1} = \phi_t + r\,\frac{\Delta t}{T}$$

where $T$ is the movement’s normal length, $\Delta t$ is the control timestep and $r$ is the rate. At $r = 1$ the movement plays at its normal speed. At $r = 2$ it plays twice as fast, through the same poses.

Who sets the rate

In the simplest setup, the rate is fixed and the controller has to keep up with the reference motion. That makes the reference’s timing a hard ceiling: a policy that tracks it well can never be faster than it.

The alternative is to let the policy choose the rate, as one more action. The reference then fixes the shape of the movement, the order of the poses, but not its timing. Legged-robot controllers use the same idea to let a gait speed up or slow down.

One trap comes with it. If the reward is paid on every step, a slower movement collects more of it. A policy that controls its own clock can then earn more by slowing down. The fix is to scale each step’s pay by the rate, so a movement earns the same in total whatever its speed.

In this project

The rhythm of the stance has a phase, and so does each punch. Until the speed work of 18 October, each punch’s clock ran at the animation’s own pace. From then on the policy sets it, from 1 to 2 times that pace. Its first version allowed half speed too, and the policy slowed every punch (the left hook from 0.52 to 0.92 s), because the per-step style pay was worth more than the quickness bonus. With the clock limited to speeding up and the per-step pay multiplied by the rate, the hooks reached the target in 0.36 s instead of 0.48–0.52. See Faster punches.