Muscle activation

A muscle doesn’t respond to the nervous system instantly. The command it receives, the excitation $u$, is a number from 0 (no signal) to 1 (all out). The muscle’s activation $a$, how much of its force it can produce right now, chases the excitation with a delay:

$$\dot a = \frac{u - a}{\tau}, \qquad \tau = \begin{cases} \tau_{\text{act}} \approx 10\ \text{ms} & u > a \ \text{(switching on)} \\ \tau_{\text{deact}} \approx 40\ \text{ms} & u \le a \ \text{(letting go)} \end{cases}$$

Switching on is faster than letting go. The force the muscle makes is then its activation times its peak force, scaled by how stretched it is and how fast it moves (see force–length–velocity).

This delay is one of the things that makes a muscle harder to control than a motor. A servo responds within a simulation step; a muscle needs tens of milliseconds, and a controller that ignores that will always be late.

In this project

MuJoCo’s muscles use exactly this model with the 10 ms and 40 ms time constants. In Muscles instead of motors the policy outputs one excitation per muscle, 50 times a second, through a smoothing filter. With the servo body’s filter (about 80 ms of extra lag) on top of the activation delay, the muscle body never learnt the stance; with a lighter one (about 20 ms) it solved in 26.6–37.8 M steps. The activations are also part of what the policy observes: 86 extra numbers on the stage 1 body.