Machine learning

In ordinary programming a person writes the rules. In machine learning a person writes a model with adjustable numbers, called parameters, and a way to measure how well it does. An algorithm then tunes the parameters until the model does well.

There are three main kinds:

• Supervised learning. Learn from examples with the right answers: photos labelled “cat” or “dog”, or houses with their sale prices. The model learns to predict the label.
• Unsupervised learning. Find structure in data with no labels, such as grouping similar customers.
• Reinforcement learning. Learn by acting and getting a reward. There are no right answers, only better and worse outcomes.

Most modern machine learning uses neural networks trained by gradient descent on a loss function. When the network has many layers, this is called deep learning.

Learning as optimisation

Under the hood, nearly all machine learning is the same move: write down a number that measures how wrong the model is, and adjust the parameters $\theta$ to make it smaller.

$$\theta^* = \arg\min_\theta \; L(\theta)$$

What changes between methods is what $L$ measures and where the data comes from.

In this project

We use reinforcement learning, so there is no dataset of correct movements. The body learns from its own attempts in simulation, guided by a reward and by a coach watching the results.