Floating point

A computer can’t store most real numbers exactly. It keeps a fixed number of significant digits and an exponent, like scientific notation: $1.2345678 \times 10^{-3}$. A 32-bit number (single precision, float32) keeps about 7 significant decimal digits; a 64-bit one (double precision, float64) about 16.

Every sum is rounded to the nearest number that fits, so two programs that do the same maths in a different order can give results that differ in the last digits. In a simulation those tiny differences grow over time, the way a nudge to a falling pencil changes where it lands. So “the same” physics on two machines means the same to within rounding, not bit for bit.

GPUs are fastest in 32-bit. Many data-centre GPUs are sold on lower precision (16-bit and below) for neural networks, which doesn’t help a physics simulation.

In this project

The CPU simulator works in 64-bit, MuJoCo Warp on the GPU in 32-bit. In Training on a rented GPU, the new mass-matrix factorisation was checked against the stock one on the same episode: joint angles matched to a millionth of a radian after 1–5 steps and 5e-5 after 10, while nudging the stock run’s start by a millionth already moved it 3e-4 after one step. So the two agree as closely as 32-bit allows.