Mean squared error

For predictions $\hat y_i$ and targets $y_i$, the mean squared error is

$$\text{MSE} = \frac{1}{n} \sum_{i=1}^{n} \left( \hat y_i - y_i \right)^2$$

Squaring makes every difference positive and makes large differences count much more than small ones. A miss of 0.2 costs four times a miss of 0.1. It is smooth, so gradient descent can follow it: its gradient with respect to a prediction is just $2(\hat y_i - y_i)/n$.

As a loss it pulls predictions towards the targets. As a measurement it says how close two models’ outputs are.

In this project

Distillation uses it both ways. The student’s loss includes the squared difference between its mean action and the teacher’s, over the 48 motor outputs. The trainer also logs the student’s mean squared error against each teacher, per action, over the states that teacher is in charge of. In the first distillation run, the error against the base model stayed at 0.002 while the body straight’s fell from 0.028 to 0.005 and the knees’ from about 0.055 to 0.008 by 39 M steps (see Shadow clones).