Glossary · Reinforcement learning
Environment
Also called env
Everything the agent interacts with: it takes the agent's action, moves the world forward one step, and returns the next observation and a reward.
In reinforcement learning, the environment is the world outside the agent. In code it is usually an object with two methods:
reset()starts a new episode and returns the first observation.step(action)applies the action, advances time, and returns the next observation, the reward, and whether the episode has ended.
The environment decides what the agent sees, what it may do, how it is scored and when an episode ends. Designing it is a large part of the work. Most of the fixes described in our posts are environment changes (when recovery mode starts and stops, how shoves are drawn, what counts as a fall), not changes to the learning algorithm.
In this project
Each environment wraps a MuJoCo simulation of one fighter on a gym mat. On every step it:
- adds the policy’s corrections to the reference pose and sends the result to the servos,
- runs ten physics steps of 2 ms (see timestep),
- applies any scheduled shove,
- works out the reward, the balance gate and whether the body has fallen,
- builds the 193-number observation.
There are several environments: the stance env, the push-recovery env (scripted recovery moves) and the survive env. Training runs hundreds of copies at once, 8 per worker process across 52 workers.