Z-score

Raw numbers are hard to compare when each has its own scale. A z-score puts them on a common one: subtract the value’s usual mean $\mu$ and divide by its usual spread, the standard deviation $\sigma$:

$$z = \frac{x - \mu}{\sigma}$$

A z-score of 1 is one standard deviation from the mean. For a bell-shaped quantity, about 68 % of values fall within 1, 95 % within 2 and 99.7 % within 3, so a z-score above 3 rarely happens by chance.

In this project

To decide when a neuron is “lit”, the torque and neuron post records each of the second layer’s 512 neurons through 3 s of quiet standing, then measures its distance from that resting level, ignoring direction:

$$z_i(t) = \frac{\left|h_i(t) - \mu_i\right|}{\sigma_i + 0.05}$$

The 0.05 stops a neuron that barely moves at rest from dividing by nearly zero. Over 3 counts as lit.