Physics simulation

A physics engine holds the state of a world (positions and velocities of everything in it) and moves it forward by a small step of time, again and again. Each step:

1. Collision detection: find which shapes are touching (see collision geometry).
2. Forces: gather gravity, motor torques, springs and outside pushes.
3. Contacts: work out the forces at each contact so that bodies don’t pass through each other (see soft contact).
4. Integration: use the net forces to update velocities, then positions.

The integration step applies Newton’s second law, $F = m a$. For an articulated body it takes a matrix form, where $q$ holds every joint angle:

$$M(q)\,\ddot{q} + c(q, \dot{q}) = \tau + J^\top f$$

$M$ is the mass matrix, $c$ gathers gravity and the forces from motion, $\tau$ is the motor torques and $J^\top f$ is the contact forces mapped onto the joints. The simplest integrator then steps forward with time step $h$:

$$\dot{q}_{t+h} = \dot{q}_t + h\,\ddot{q}_t, \qquad q_{t+h} = q_t + h\,\dot{q}_{t+h}$$

Why train in simulation

• It’s safe: a simulated fighter can fall thousands of times.
• It’s fast: many copies run in parallel, much faster than real time.
• It’s measurable: every force and position is known exactly.

Sim-to-real

The catch is that a simulation is never exactly the real world. A policy can exploit quirks of the simulator that don’t exist in reality. Moving a policy onto a real robot, sim-to-real transfer, usually needs extra work, such as randomising masses and friction during training so the policy can’t rely on exact values.

In this project

We use MuJoCo. The fighter isn’t aimed at a real robot, so sim-to-real isn’t the goal. What matters is that the physics is close enough to a human’s for a coach to judge the movement.