Multilayer perceptron

A multilayer perceptron (MLP) is a neural network made only of fully connected layers. Data flows one way, from input to output, with no memory between calls.

An MLP with two hidden layers computes

$$y = W_3\,\sigma\big(W_2\,\sigma(W_1 x + b_1) + b_2\big) + b_3$$

“2 × 256” means two hidden layers, each with 256 units.

Counting weights

A fully connected layer from $n$ inputs to $m$ outputs has $n \times m$ weights plus $m$ biases. The stance policy on the muscular body maps 193 inputs to 43 outputs:

layer	weights + biases
193 → 256	49,664
256 → 256	65,792
256 → 43	11,051
total	126,507

Why an MLP

For control from a fixed-size state, an MLP is usually enough. It’s fast, easy to train, and has no hidden state to manage. Other designs add things an MLP lacks: a recurrent network (RNN, LSTM) remembers past inputs, and a transformer can attend over a long history. Here the needed history (previous actions, filter state, rhythm phase) is put in the observation, so the MLP doesn’t have to remember it.

In this project

The actor and critic are separate 2 × 256 MLPs with ELU activations.