Glossary · Machine learning
Model capacity
Also called capacity, network size, the net is full
How much a network can represent. A network that is full can only learn something new by getting worse at something else.
A network’s capacity is, loosely, how complicated a function it can represent. It grows with its number of units and layers. A network with too little capacity for its task underfits: it can’t get the training error down however long it trains. When one network has to learn several skills, a lack of capacity shows up as competition. Improving one skill makes another worse, because they share the same weights.
There is no simple number for “how full” a network is. One practical sign is a training error that stops falling on the newest skill while the old ones stay low. The usual fix is to make the network wider, ideally without changing what it already does (function-preserving growth).
In this project
The fighter’s policy is a multi-layer perceptron with two hidden layers of 512 units. Before distilling three new techniques into it, the plan was to widen it if the distillation error stalled high on the new ones. It didn’t: their errors were still falling at the end of the first run (body straight 0.003, knees about 0.005), and the base’s stayed at 0.002. The trade-offs that did show up, like the blocks against the cross in the air, came from the mix of practice and the teacher weight, not from a full network (see Shadow clones).