Glossary · Machine learning
Frame stacking
Also called stacked history, history window, stacked frames, observation history
Giving a network the last few frames of its input side by side, so it can see motion without having a memory of its own.
A plain feed-forward network such as an MLP has no memory: it sees only the input of this moment. If that input is a snapshot, it can’t tell a still arm from one that has just started moving.
Frame stacking is the simplest fix. Alongside the current frame, the network gets the previous few frames too: the input at time becomes
From the differences between them, the network can work out speed and acceleration. It is how the early Atari game-playing networks saw which way a ball was moving.
The window is fixed, so anything older than its oldest frame is forgotten. For longer patterns, such as an opponent’s habits over a round, a network with its own memory is needed: a recurrent network such as a GRU. That costs more training time and tuning, so stacking comes first when the window is enough.
In this project
In the blocking post, each of the opponent’s keypoints comes with its velocity and its positions 40 and 80 ms earlier. That window covers the start of a punch, which is all a single block needs. A GRU is planned for later, for feints, set-ups and whole fights.