Glossary

The machine learning, robotics and physics simulation terms used on this site, explained. Terms underlined with dots in a post link here.

A

Actionaction space, continuous control: What the policy outputs on each step, such as a target angle for every motor. Reinforcement learning
Action clippingclipping, saturation, action bound: Cutting off actions that fall outside the allowed range. It's needed for safety, but it can hide drift in the policy's outputs. Reinforcement learning
Activation functionactivation functions, nonlinearity, ELU, ReLU, tanh: The simple curve each neuron applies to its weighted sum of inputs; without it, a stack of layers would collapse into one straight-line function. Machine learning
Actor-criticactor, critic: An RL design with two learned parts: the actor (the policy) chooses actions, and the critic (a value function) judges how good things are going. Reinforcement learning
Actuatormotor, actuated joint: Anything that makes a robot or simulated body move, such as a motor at a joint. Robotics and control
All-reduceallreduce, gradient averaging, learner process, learners: A step where several processes each add in their own numbers and all get back the same total, used to keep copies of a network in sync. Computing
AnthropometryANSUR II, de Leva, segment parameters, body segment parameters: The measurement of the human body: sizes, proportions, and the mass and inertia of each body segment. Used to build realistic simulated bodies. Biomechanics
Auxiliary lossimitation loss, supervised loss, auxiliary task, button imitation: An extra training signal added beside the main reward, such as a label saying what the right answer was, to speed up learning. Machine learning

B

Behaviour cloningbehavior cloning, imitation learning: Training a policy by supervised learning to copy an expert's recorded actions, state by state. Machine learning
bfloat16bf16, brain floating point, mixed precision, AMX, autocast: A 16-bit number format that keeps the range of ordinary 32-bit numbers but only about three significant digits, so maths on it runs much faster. Computing

C

Camera calibrationcalibration, calibrate, calibrated, checkerboard, ChArUco board, intrinsics, extrinsics: Measuring a camera's lens and where it stands and points, so a pixel in its picture can be turned into a ray in 3D space. Robotics and control
CAPS regularisationCAPS, Conditioning for Action Policy Smoothness: Extra loss terms that make a policy's actions smooth: similar from one step to the next, and similar for similar states. Reinforcement learning
Capture pointinstantaneous capture point, divergent component of motion, DCM: The spot on the floor where a moving body would have to step to come to a complete stop. If it's outside the feet, the body must step or fall. Robotics and control
Catastrophic forgettingforgetting, catastrophic interference: When a neural network trained on something new loses a skill it already had, because the same weights are reused for both. Machine learning
Centre of massCOM, CoM, center of mass, centre of gravity: The average position of a body's mass, weighted by how much mass is where. Gravity acts as if it pulls on this one point. Physics simulation
Checkpointsnapshot, saved model, frozen policy: A saved snapshot of a model during training, so training can resume after a stop or an earlier version can be used again. Machine learning
Co-contractionagonist, antagonist, antagonist muscle, agonist muscle, muscle pair, opposing muscles, reciprocal inhibition: Muscles on both sides of a joint pulling at once. It makes the joint stiff without moving it, and brakes any fast movement. Biomechanics
Coding agentAI agent, agent, AI harness, agent harness: An AI assistant that doesn't just answer questions but writes code, runs it, reads the results and keeps going, on a real computer. Machine learning
Collision geometrycollider, geom, capsule, collision shape: The simple shapes a physics engine uses to work out what is touching what: spheres, capsules, boxes and meshes. Physics simulation
Confidence intervalconfidence intervals, 95 % interval, error bar, error bars, Agresti–Caffo, uncertainty: A range around a measured number that shows how far off it could be because of chance: a wide interval means the measurement is not very precise. Machine learning
Cosine similaritycosine, cosine distance, similarity matrix, fingerprint: A measure of how alike two lists of numbers are in pattern, ignoring their size: 1 for the same direction, 0 for unrelated, −1 for opposite. Machine learning
Covariate shiftdistribution shift, compounding errors: When a model meets inputs unlike the ones it was trained on. A copied policy that drifts off the expert's path has never seen where it ends up. Machine learning
CUDA graphCUDA graphs, graph capture, graphed, torch.cuda.graph: A recording of a sequence of GPU kernels that can be replayed as one unit, so the CPU doesn't have to launch each kernel again. Computing
Curriculum learningcurriculum, automatic curriculum: Training on easy versions of a task first and making it harder as the agent improves, like a coach raising the level for a student. Reinforcement learning

D

Decoupled PPOasynchronous PPO, batch size-invariant PPO: A variant of PPO that lets data collection run on a slightly older copy of the policy while the learner updates, so the two can overlap. Reinforcement learning
Degrees of freedomDoF, DOF, joint, hinge joint, ball joint: The number of independent ways something can move. Each simple joint adds one; a free body floating in space has six. Robotics and control

E

Early stoppingplateau, stopping rule: Ending a training run once it stops improving, instead of running for a fixed time. Machine learning
Embarrassingly parallelparallel training, trivially parallel, running in parallel: A job that splits into pieces that don't need each other, so each piece can run on its own machine at the same time. Computing
Environmentenv: Everything the agent interacts with: it takes the agent's action, moves the world forward one step, and returns the next observation and a reward. Reinforcement learning
Environment stepstep, sample, timesteps, M steps: One tick of the agent–environment loop: one observation, one action, one reward. Training length is counted in these steps. Reinforcement learning
Episodetrajectory, termination: One attempt at the task, from reset to the end: a fall, a time limit or success. Reinforcement learning
Evaluation gategate, the gate, evaluation, deterministic evaluation: A fixed set of tests a trained model must pass before it is kept: here every skill, played without exploration noise, against fixed seeds. Machine learning

F

Feed-forward controlfeed-forward, feedforward, servo feed-forward, velocity feed-forward: Telling a motor in advance what a planned movement will need, instead of waiting for an error to build up and then correcting it. Robotics and control
Fine-tuningwarm start, warm-starting, resume, from scratch: Continuing to train an already-trained model, often on a new or harder task, instead of starting from scratch. Machine learning
Floating pointfloat32, float64, 32-bit, single precision, double precision, rounding error, FP32: How computers store real numbers: a fixed number of significant digits plus an exponent, so every result is rounded slightly. Computing
FLOPSFLOP, TFLOPS, teraflops, GFLOPS, peak, compute-bound: Floating-point operations per second: how many additions and multiplications of real numbers a processor does in a second. Computing
Force–length–velocityforce-length curve, force-velocity curve, force-length relationship, force-velocity relationship, eccentric contraction, lengthening muscle, optimal fibre length: A muscle's force depends on how stretched it is (strongest near its best length) and how fast it moves: weaker while shortening fast, stronger while being stretched. Biomechanics
Frame stackingstacked history, history window, stacked frames, observation history: Giving a network the last few frames of its input side by side, so it can see motion without having a memory of its own. Machine learning
Frictioncoefficient of friction, skating, friction cone: The force that resists surfaces sliding over each other. It's what lets feet push on the floor without skating. Physics simulation
Function-preserving growthNet2Net, growing a network, widening, zero-initialised inputs: Adding inputs or units to a trained network in a way that leaves its outputs unchanged, so training can continue from exactly where it was. Machine learning

G

Gaussian kernelGaussian reward, Gaussian term, radial basis function, exp(-error²): A bell-shaped score, 1 when an error is zero and falling smoothly towards 0 as it grows; a common way to turn an error into a reward. Reinforcement learning
GPUgraphics card, CUDA, graphics processing unit: A graphics processor: a chip with thousands of small cores that do the same sum on lots of numbers at once. Fast for neural networks, less useful for CPU physics. Computing
GPU kernelkernel, kernel launch, launch, CUDA kernel, thread block: One small program sent from the CPU to the GPU, run by thousands of GPU threads at once; every launch has a small fixed start-up cost. Computing
Gradient descentSGD, stochastic gradient descent, Adam, backpropagation, optimiser: The basic method for training neural networks: repeatedly nudge every parameter a small step in the direction that reduces the loss. Machine learning

H

Half precisionfloat16, FP16, 16-bit float, reduced precision: Storing a number in 16 bits instead of the usual 32: about three significant digits, half the memory. Computing
Hidden layerhidden layers, neuron, neurons, hidden unit, unit, layer, activations: A layer of a neural network between its inputs and its outputs. Each of its units, or neurons, mixes all of the previous layer's numbers into one new number. Machine learning
Hill climbinglocal search, random search: A simple search method: make a small random change, keep it if it improves the score, and repeat. Machine learning
Human-in-the-loopHITL, human feedback: Training where a person judges results and steers what happens next, instead of relying only on an automatic score. Machine learning
Hyperextensionhyperextend, hyperextended, ligament, ligaments, joint lock: A joint straightened past its normal range. Ligaments and bone stop it; pushed further, they tear or break. Biomechanics
Hyperparameterhyperparameters, tuning: A setting chosen by the person training the model, such as the learning rate or network size, rather than learned during training. Machine learning

I

Inertiamoment of inertia, inertia tensor, rotational inertia: How hard it is to change an object's motion. For rotation, the moment of inertia depends on how far the mass is from the axis. Physics simulation
Interpolationinterpolate, interpolated, extrapolation, extrapolate, aim table, aim grid, lookup table: Estimating values between known ones; going past the known range is extrapolation, and is much less reliable. Machine learning
Interpretabilityexplainability, mechanistic interpretability, explainable AI, XAI, probing, ablation: Looking inside a trained neural network to understand what its parts do and why it acts as it does, instead of judging it only by its behaviour. Machine learning
Inverse kinematicsIK: Working out the joint angles that put a hand or foot at a chosen position, the reverse of forward kinematics. Robotics and control

J

Jitterchatter, twitching, direction reversals: Small, fast, back-and-forth twitching of the joints. Common in RL policies; we measure it as direction reversals per joint per second. Robotics and control

K

Kernel fusionfused kernel, fusion, torch.compile, compile, compiled, compiler, Triton: Merging many small GPU programs into a few bigger ones, so the GPU spends less time starting programs and more time computing. Computing
Keyframekeyframing, keyframed motion, interpolation: A pose set at a specific moment in a motion. The frames between keyframes are filled in automatically by interpolation. Robotics and control
Keypointkeypoints, pose estimation, skeleton, landmarks: A tracked point on a body, such as a wrist, elbow or hip, used to describe a pose with a few numbers instead of a whole image. Robotics and control
Kinematicsforward kinematics, kinematic, dynamics: The geometry of motion: where each body part is, given the joint angles, without asking what forces caused it. Robotics and control
Kinetic chainkinematic chain, power chain, hip drive: The linked segments of the body (feet, legs, hips, trunk, shoulder, arm) that pass force and speed from one to the next in a movement such as a punch. Biomechanics

L

Latencyvision delay, reaction time, delay, lag: The delay between something happening and the system knowing about it; for a fighter, the reaction time. Robotics and control
Learning ratestep size, η, lr: How big a step training takes each time it updates the weights. Too big and training becomes unstable; too small and it's slow. Machine learning
Local optimumlocal optima, local minimum, local maximum, plateau: A solution that is better than everything close to it but not the best overall, so small improvements can't get away from it. Machine learning
Loss functionloss, objective, cost function: A number that measures how wrong a model is. Training adjusts the model to make it smaller. Machine learning
Low-pass filteraction filter, exponential moving average, EMA, smoothing filter, cutoff frequency: A filter that lets slow changes through and smooths away fast ones. Used on a policy's actions to stop twitching. Robotics and control

M

Machine learningML: Getting computers to do a task by learning patterns from data or experience, rather than by following rules a programmer wrote by hand. Machine learning
Markov decision processMDP, discount factor, return: The maths framework behind reinforcement learning: states, actions, rewards, and rules for how one state leads to the next. Reinforcement learning
Mass matrixinertia matrix, factorisation, factorization, Cholesky, LDL: A table that says how hard each joint is to accelerate, given everything attached to it; the physics has to factorise it every step. Physics simulation
Mean squared errorMSE, squared error, L2 loss: The average of the squared differences between predictions and targets: a common way to measure how far one set of numbers is from another. Machine learning
Metabolic costmetabolic energy, energy cost, energy expenditure, muscle energy: The chemical energy muscles burn to make force and movement. It is what makes a human tired, and what a motor's electricity bill doesn't capture. Biomechanics
Mixture of expertsmixture of experts, MoE, experts, gating network, MCP: Several specialised networks plus a small one that decides, moment to moment, how much to listen to each. Machine learning
Model capacitycapacity, network size, the net is full: How much a network can represent. A network that is full can only learn something new by getting worse at something else. Machine learning
Moment armlever arm, muscle moment arm: A muscle's lever on a joint: how far its line of pull passes from the joint's axis. Its force times its moment arm is the torque it makes. Biomechanics
MomentumN s, N·s, linear momentum, impact: Mass times velocity: how much motion something carries. A punch's momentum is what it can pass on to the target. Physics simulation
Monotone cubic splinePCHIP, Fritsch–Carlson interpolation, cubic Hermite spline, spline: A smooth curve through a set of points that never overshoots them, so it rises where the points rise and stays flat where they're flat. Robotics and control
Motion capturemocap, stock motion, motion clip, animation clip: Recording a real person's movement, usually with markers or cameras, so a computer can replay it on a digital body. Biomechanics
Motion plannerplanner, footstep planner, step planner: The part of a system that decides which movement to make next to reach a goal, leaving the details of making it to a controller. Robotics and control
Motion retargetingretargeting, retarget, retargeted: Moving a motion recorded on one body onto another body with different proportions, keeping what matters about the movement. Biomechanics
MuJoCoMulti-Joint dynamics with Contact: A fast, accurate physics engine for simulating bodies with joints and contacts, widely used in robotics and reinforcement learning research. Physics simulation
MuJoCo WarpMJWarp, mujoco-warp, Warp, NVIDIA Warp, GPU physics: A version of the MuJoCo physics engine that runs on a GPU and simulates thousands of copies of a world at once. Physics simulation
Multilayer perceptronMLP, fully connected network, feed-forward network, 2 × 256: The simplest kind of neural network: a stack of fully connected layers, where every unit in one layer feeds every unit in the next. Machine learning
Muscle activationactivation, excitation, activation dynamics, muscle excitation, neural excitation: How switched on a muscle is, from 0 to 1. It follows the nerve's command (the excitation) with a delay: about 10 ms to switch on, 40 ms to let go. Biomechanics
Muscle redundancyredundancy, redundant muscles, actuator redundancy, muscle sharing problem: A body has more muscles than it has ways to move, so many different patterns of muscle effort give the same movement. Biomechanics
MyoSuiteMyoLeg, MyoLeg26, myo_sim: An open-source set of musculoskeletal models and tasks for MuJoCo, built for research on learning muscle control. Its leg models are MyoLeg26 and MyoLeg. Physics simulation

N

Neural networkartificial neural network, deep learning, weights: A function built from layers of simple units, each multiplying its inputs by learned weights, adding them up and passing the result through a nonlinearity. Machine learning
Newton (unit)N, force, impulse: The SI unit of force. One newton accelerates one kilogram by one metre per second, every second. A 77 kg person weighs about 755 N. Physics simulation

O

Observationstate, observation space, observation normalisation: The numbers the policy sees at each step, describing the state of the body and the task. Reinforcement learning
Occlusionline of sight, ray casting, blindness, occluded, hidden: When something is hidden from view because something else is in the way, such as your own arms in front of your eyes. Robotics and control
On-policyoff-policy, on-policy distillation, DAgger: Learning from experience gathered by the policy being trained, rather than from someone else's recordings or from old versions of itself. Reinforcement learning
Open-loop controlopen loop, closed-loop control, closed loop, feedback control: Sending fixed commands without looking at what happens. The opposite of feedback (closed-loop) control, which corrects itself from what it measures. Robotics and control
Oracleoracle baseline, upper bound, perfect reader: A hand-made baseline that is given information or abilities the learner doesn't have, to show the best result possible. Machine learning

P

Parallel environmentsvectorised environments, vectorized environments, copies of the world, worlds, num envs, batch of environments: Many copies of the same training world run side by side, so one policy collects experience from all of them at once. Reinforcement learning
Phase variablephase, motion clock, punch clock, clock, time scaling: A clock that says how far through a movement the body is, from 0 at the start to 1 at the end, so a motion can be played faster or slower. Robotics and control
Physics simulationphysics engine, simulator, rigid-body dynamics, sim-to-real: Computing how objects move under forces, one small time step at a time, so a virtual body behaves like a real one. Physics simulation
Policyπ, controller, actor: The agent's decision rule: given what it observes, which action it takes. In deep RL, the policy is a neural network. Reinforcement learning
Policy distillationdistillation, teacher, teacher policy, student, anchor, multi-teacher distillation, one-technique teacher: Training one policy to copy another's actions. Here a frozen copy of an older policy acts as a teacher, so new training doesn't wipe out skills it already had. Reinforcement learning
Position servoPD controller, PD control, servo, position actuator, stiffness, damping: A motor controller that pushes a joint toward a target angle, with a force that grows with the error, up to a limit. Robotics and control
Profilingprofiler, profile, bottleneck, torch profiler: Measuring where a program spends its time, part by part, to find what to speed up. Computing
Proximal Policy OptimizationPPO: A widely used reinforcement learning algorithm that improves a policy in small, safe steps by limiting how far each update can move it. Reinforcement learning
Push recoveryshove recovery, perturbation, balance recovery: Staying on your feet after an unexpected shove, by leaning, stepping or hopping. Robotics and control

R

Random seedseed, seeds, random seed: The number that starts a program's random number generator. Two training runs that differ only in their seed show how much of a result is luck. Computing
Range of motionROM, joint limits, joint range: How far a joint can move, in degrees, from one end of its travel to the other. Biomechanics
Recurrent neural networkRNN, GRU, LSTM, gated recurrent unit, memory: A neural network with a memory: it carries a hidden state from one step to the next, so it can use what it saw earlier. Machine learning
Reference motionmotion tracking, motion imitation, reference, tracking reward: A target motion the policy is rewarded for following, such as a stance, a rhythm or a keyframed step. The policy adds the physics. Robotics and control
Reference state initialisationRSI, reference state initialization, mid-motion starts, mid-walk starts: Starting some training episodes partway through a reference motion, in its pose and speed, so the policy practises every part of it and not only the beginning. Reinforcement learning
Regularisationregularization, weight decay: Anything added to training that prefers simpler or smoother solutions, usually an extra penalty in the loss. Machine learning
Reinforcement learningRL: A way of training a program by trial and error: it acts, gets a score called a reward, and slowly learns to act in ways that score more. Reinforcement learning
Residual policyresidual action, residual control: A policy that outputs corrections to a reference motion rather than whole new motions, so it only learns the difference. Reinforcement learning
Reward functionreward, reward design: The rule that scores each step of the agent's behaviour. The agent learns to get as much reward as it can, so the reward defines the task. Reinforcement learning
Reward hackingspecification gaming, reward exploitation: When an agent finds a way to score well that isn't what the designer meant, exploiting a gap in the reward. Reinforcement learning
Reward shapingpotential-based shaping, shaping reward: Adding extra reward terms to guide learning. Potential-based shaping does this in a way that provably doesn't change which policy is best. Reinforcement learning
Rolloutbatch, horizon, iteration: Running the current policy in the environment for a fixed number of steps to collect experience for one training update. Reinforcement learning

S

Shared memorymultiprocessing.shared_memory, pipes, inter-process communication, IPC: A block of memory that several processes can read and write directly, so data passes between them without being copied. Computing
Shortcut learninggiveaway, giveaways, spurious correlation, Clever Hans, tell, data leak: When a model learns an easy cue that happens to predict the answer in training, instead of the thing you meant it to learn. Machine learning
Sign testsign tests, binomial test, p-value, p value: A simple test of whether one option wins more often than chance across many paired comparisons, counting only who won each one. Machine learning
Soft contactcompliance, contact model, penetration, solref, solimp: Modelling contacts as slightly springy, so surfaces can sink into each other a little. Used both for stability and to model soft things like skin and foam. Physics simulation
Spot instanceEC2 spot, spot pricing, on-demand: A cloud computer rented cheaply from spare capacity, which the provider can take back at short notice. Computing
Statistical powerpower, underpowered, sample size, power analysis: The chance that a test will detect a real difference of a given size; a test with little power often finds nothing even when something is there. Machine learning
Stochastic policyGaussian policy, deterministic policy, exploration noise, sampled actions: A policy that outputs a probability distribution over actions and samples from it, so the agent explores. Taking the most likely action instead makes it deterministic. Reinforcement learning
Stretch reflexmyotatic reflex, reflex, spinal reflex: A spinal reflex that excites a muscle when it is stretched, pulling the limb back toward where it was going without waiting for the brain. Biomechanics
Support polygonbase of support, support region: The outline on the floor around all the points where the body touches the ground. Standing still, the body balances if its centre of mass is above it. Robotics and control

T

Tendon pathspatial tendon, muscle path, via point, wrapping surface, muscle wrapping, line of action: The route a simulated muscle takes over the bones: anchored at its origin and insertion, through via points, and wrapped round bones, so its length and lever change as the joints move. Physics simulation
Timesteptime step, physics step, control frequency, substeps, Hz: The small slice of time a physics simulation advances in one step. The policy usually acts less often, once every several physics steps. Physics simulation
Torquemoment, N·m, newton-metre: A twisting force: what makes something rotate. A motor turns a joint by applying torque, measured in newton-metres. Physics simulation
Training divergenceNaN, collapse, policy collapse, instability: When training breaks down instead of improving: the loss blows up, the numbers become NaN, or a good policy suddenly collapses. Machine learning
Triangulationtriangulate, triangulated, stereo: Finding a point's position in 3D from where it appears in two or more calibrated cameras. Robotics and control
Trunk leanlean, lean back, pitch, trunk pitch, arch, arched back, posture: How far the pelvis or chest tips forward or back from upright, measured as an angle along the direction the body faces. Biomechanics

V

Value functioncritic, V(s), advantage, GAE: An estimate of how much future reward the agent can expect from a given state. The advantage says whether an action did better or worse than that. Reinforcement learning
vCPUvirtual CPU, core, worker process, parallelism, hyperthreading, hyperthread, SMT: A virtual CPU: one unit of processor capacity in a cloud machine. A 64-vCPU machine can run about 64 things at once. Computing

Z

Z-scorestandard score, standardised, standardized, standard deviations, σ: How far a value is from its usual level, measured in units of its usual spread: 0 is typical, 3 is far out of the ordinary. Machine learning

Glossary

A

B

C

D

E

F

G

H

I

J

K

L

M

N

O

P

R

S

T

V

Z

Can we use analytics cookies?