Glossary · Reinforcement learning
Episode
Also called trajectory, termination
One attempt at the task, from reset to the end: a fall, a time limit or success.
An episode is a run of steps from reset() to the end. It ends in one of two ways:
- Termination: something happens that ends the task, like a fall. There is no future reward after it.
- Truncation: a time limit cuts it off. The task could have gone on, so the learner should still estimate the reward it would have got afterwards from its value function. Treating a time limit as a real ending is a classic bug: the agent then learns that the world ends at 10 s.
Each episode usually starts from a slightly different state, so the policy learns to cope with variety rather than one fixed situation.
Episode length as a signal
When falling ends the episode, the average episode length is a quick measure of skill. An agent that survives the full 10 s every time has stopped falling.
In this project
Episodes last 10 s at 50 steps a second, so 500 steps. Each one draws its own rhythm start phase, tempo (0.8–1.3 beats per second) and amplitude. A fall is the pelvis dropping 0.3 m, the head going below 1.1 m, or any part other than a foot touching the floor. The shove curriculum learning rises only when episodes average at least 8 of the 10 s.