Muay Thai · Footwork · Walking · Reward design · Coaching
Footwork: walking to a spot in a Muay Thai stance
Training cost about $3522 EC2 spot training runs plus about 30 short test and table-building jobs, about 26 machine-hours, 18–19 October 2026 · breakdown
By the end of the faster punches post, the fighter could throw five punches from its guard, quickly and on command. But it threw them all from the same spot. A fighter who can’t move can’t choose when to be in range.
Tony Yum, the coach, puts it like this:
Controlling the distance is one of the most important facets of striking martial arts. You want to be close enough that you are still engaged, but not close enough that the opponent can hit you. Every striking technique has slightly different range, and your opponent has different range. And so whoever understands the distance best has a huge advantage.
This post is about the first step towards that: footwork. Not the sudden step in to land a punch, or the quick step out to avoid one. That comes later. This is the slow movement at the start of a round, when two fighters are sussing each other out: moving around in a good Muay Thai stance, forward, back, sideways and diagonally, and pivoting to turn.
It covers 22 training runs over 25 hours, from 17:34 UTC on 18 October to the evening of 19 October. The model that was kept finished training at 15:44 UTC on 19 October.
The task: a spot and a facing
The fighter is given a target: a spot on the floor for the ball of its lead foot, and a direction to face. It has to get there, in its stance, and settle. The spot can be anywhere: ahead, behind, to the side, or behind it with an order to turn and face it.
As with the punches, the movement starts as a reference motion, an animation with no physics in it. This one was built from a 28-second video of Tony walking in his stance: forward, backward, a pivot to face the camera, forward and back again, and pivots back. From the video came a library of single steps: 16 directions around the body, each at any size up to a full step (26 cm forward, 22 cm back, 20 cm to the side), plus pivots to turn left and right. Going forward, the lead foot steps first and lands on its ball; the rear foot follows, low. Going back, the rear foot steps first. Each step is timed to one beat of the fighter’s rhythm, the gentle weight shift it makes in its stance, so the walk keeps the same pulse as everything else it does. The foot and hip positions are scripted, and inverse kinematics works out the joint angles.
Something has to decide which step to take next. That is a simple planner, like the ones that move characters in a video game. At the start of each beat, it looks at where the target is. If the facing is more than 30° off, it pivots. Otherwise it steps straight towards the spot, in whatever direction that is, sized to land on it. The policy, the neural network that drives the body, has to turn each planned step into a real one, with real muscles and gravity, while the fighter is being shoved now and then.
And it must not forget its punches. A frozen copy of the best punching model, the one from the faster punches post, acts as a teacher (policy distillation): whenever the fighter isn’t walking, the new policy is pulled towards the teacher’s actions. Walking is free to change; the punches should not.
Lesson 1: a pivot can’t twist a loaded foot
Each run continued from the model before it (fine-tuning), starting from the punching model. The first run learnt to step, but in 8.8 M steps of training it never once turned the body in a pivot. Eight pivots in a row, the hips turned 0° of 45°.
The reference had spun the lead foot on its ball while the weight was on it. Real feet can do that on a canvas, but the simulated foot’s contact with the floor doesn’t slip when twisted. And the lead hip can’t rotate 45° over a planted foot anyway. The reference was asking for something the body couldn’t do.
The second run used a different pivot: all the weight onto the flat lead foot, the rear foot lifts and swings round while the hips turn over the planted foot, the rear foot lands, and then the lead foot lifts a little and re-aims. 30° per beat. Within 6 M steps, pivots turned 21–23° each. But now the steps had stopped working: the lead foot hardly moved. The rewards for walking were simply too small. Getting 26 cm nearer the target paid about 0.5 per beat, against about 50 per beat for holding a good stance. The third run made the walking rewards several times stronger, and the steps came back, slowly.
Lesson 2: tell the muscles in advance
Steps were landing, but short: a forward step covered 4 % of the planned distance, a sideways one under 20 %. Two changes fixed most of it.
Each joint is driven by a position servo, a motor that pulls the joint towards a target angle. A servo lags behind a target that keeps moving. So the walk’s servo targets now lead the reference by that lag, a kind of feed-forward: the muscles are told where the leg will need to be, not where it is now. And 30 % of training episodes now started partway through a step, in the reference’s pose and speed (reference state initialisation), so the policy practised the middle of a step and not only its start.
One run later, steps covered 92 % of the plan going back, 58 % forward and 56 % sideways. Six of eight test targets were reached. The two misses circled round the spot, 10–15 cm away, taking small steps that overshot. The planner was changed so that a spot, once reached, stays reached. From then on, every test target was reached, 8 out of 8, with no falls.
Lesson 3: don’t forget the punches
By then the fighter had walked for about 100 M steps, and the punch test said it had lost something. The punching model scored 38 of 60 presses on its own test; the walker scored 20. The teacher’s pull (weight 2) had let the punches drift.
Raising the teacher’s weight to 10 didn’t help, and the reason was a bug. A neural network sees its inputs scaled to a standard range, using running averages of what it has seen so far (observation normalisation). The teacher was being fed inputs scaled with the student’s averages, which had drifted as walking data came in. The teacher’s advice was off by about 0.007 per motor, sometimes 0.025. That is small, but it was the same size as the differences that made the punches miss. Pulling harder towards slightly wrong advice can’t fix anything. With the teacher on its own scaling, the punches came back.
The test itself also had a flaw. Each test pressed every punch six times, at six moments in the fighter’s rhythm. But the walking environment drew its random numbers in a different order, so its six presses fell at different moments, and six presses swing the score by a lot. The punching model itself scored 19 of 60 in the walking environment, against 38 in its own. With the moments fixed and 20 presses per punch, the punching model scored 112 of 200, and the walker after the fix 107. The loss before the fix had been real, but about 10 points, not 30.
Lesson 4: steer with the steps, not with the rewards
The walk reached every target, but slowly: about 7.5 s per target. Measuring each step showed where the time went. The hips drifted 3–10° per step, always the same way for a given direction, until the facing was so far off that a pivot was needed, and pivots moved the lead foot 5–13 cm.
The first fix asked the fighter to steer up to 10° per step through the rewards alone. It didn’t learn it, and got slower: the reference’s joint angles were still those of a straight step, so the rewards for matching the pose held the hips straight. The second fix put the steering into the reference: 32 new steps, each a normal step that also turns 10° left or right while the foot is in the air. That worked at once. Forward steps reached their full length (27–28 cm, against 17–22), and a target took 6.7 s instead of 7.4.
Tony asked for one more thing here: make sure training includes turning all the way round, to face a target behind. It already did (about 11 % of training targets needed a turn of more than 150°), but turning round now took the longest: five or six pivots of about 33°. So the left pivot was made to turn 60°, the hips turning half over the planted lead foot and half over the rear foot once it lands. The right pivot had to stay at 30°: turning right, the rear hip runs out of rotation as soon as the foot swings past the hips. The planner turns the way that needs fewer pivots. A 180° turn on the spot went from 5.9 s to 3.5 s (Figure 2).
Lesson 5: Tony’s eye for style
At this point, the walk worked. It reached all 40 test targets (ten kinds of target, four times each, with shoves) in 6.09 s on average, landing 5.4 cm from the spot, and the punches held. Then Tony watched it as a coach.
After each step it goes back to Muay Thai stance … make the walking stay in walking mode and not flip between stance and walking.
He was right. Every step in the reference started and ended at rest in the stance, and about a quarter of each step had no foot moving. Each step was split into four versions: moving off, flowing through, arriving, and the single step. Walking on, the fighter now passed through the stance instead of stopping in it. The hips still slowed to about a quarter of walking speed at each stance, so the next run paid for the hips’ speed over the floor. That didn’t move it either. Tony saw why first:
After every step it goes into the stance again … almost like there’s 70% weight on the back leg and 30 at the front … ideally, there should be no stance at all, just from step to step.
The reference itself returned to the stance’s weight, 69 % on the rear foot, between every two steps. So the stance was taken out of the middle of a walk: the weight at 50/50 between flowing steps. Then came more of Tony’s notes, one at a time:
- “The back leg often drags and that even sometimes trips up the model.” The feet were lifted higher, with a reward for clearing the floor and a cost for scraping it.
- “First the heel lift, but then the toes should follow so that in mid-air the foot is flat rather than having the toes hanging down.” The rear foot now levels off in the air. The ankle can’t bend far enough to hold it fully flat behind the body, so it was lifted higher to get close.
- “I still see the heavy back foot weight … and the front leg with the heel off the floor.” The lead heel now stays down between steps.
- “The target destination of the walk doesn’t need to be so accurate … the punch itself should be able to adjust a little bit of distance.” A target now counts as reached within 15 cm and 15°, not 8 cm and 8°.
Each change made the walk look more like Tony’s. And each one made it harder. The higher rear foot made every step a balancing act, and the policy answered by taking shorter steps. Then:
It looks like he’s limping … he tilts almost to the front leg, his body even tilts to the left.
I still see a lot of limping, like it’s bowing diagonally forward, so the torso is bending.
One of the new rewards paid for the right share of weight on each foot. The fighter’s way of loading the front foot was to lean over it: a bow, 8.2° beyond the stance’s own lean on average and up to 16°. That is a textbook case of reward hacking: it did what the numbers asked, not what was meant. The next run removed that reward and added one for keeping the trunk upright. The bow went (4° on average), but the fighter had been using it to get its weight across. Without it, it couldn’t step left:
The diagonal step is completely wrong. It keeps stepping forward and back. It doesn’t actually move forward.
Steps to the left now covered 9–15 cm of the planned 20, with 90 % or more of the weight left on the rear foot. The fighter stalled about 28 cm from the spot, just outside the 25 cm within which the planner would have called it close enough, and shuffled its lead foot back and forth. One more reward, for the hips following the reference’s path, made it worse: 17 of 40 targets.
Function first
Tony made the call:
Okay let’s remove all the requirement for the looks. Walking to the target is the most important. Except it should not bow.
The next run went back to the best walker, the one from before the style work (40 of 40, 6.09 s per target), with its original reference. Every look reward came off: hip speed, foot height, scraping, the weight share. Two things stayed: the reward for keeping the trunk upright, and the loose 15 cm arrival.
At 20.7 M steps it reached all 40 targets, with no falls, in 6.70 s on average. Its forward bow averaged 2.8° beyond the stance, at most 6.9°. Its punches scored 60 of 120 on the 12-press test, the same as every walking run since the teacher fix. That is the model this post keeps.
A last run tried to fix the weak spot: steps sideways. The upright reward had counted any lean as a bow, including the small sideways lean a sideways step needs. So that run allowed sideways lean and banned only forward lean. Its steps got shorter everywhere, and it reached 38 of 40. The sideways lean wasn’t what held the steps back.
What Coach Tony thinks
Tony’s own summary:
It turns out it’s very difficult to learn how to walk stylistically. It went from lifting the leg at every step, like Muay Thai stance, to limping, putting too much weight on the front leg, dragging the back feet. But the more constraints [I] added on stylistically, the worse the walking becomes. So there’s a balance between style and actual practical movements.
That is what the numbers show too (Figure 4). Every one of his notes was a fair coaching point, and the fighter did change in the way he asked. But every new rule was one more thing to satisfy at once, on a body that was already busy staying upright under shoves and keeping its punches. Each rule cost a little speed or a little reach, and two of them broke the walk.
For now, function wins. The fighter gets to the spot, upright, ready to punch. Tony also pointed out that the arrival doesn’t need to be exact: the punch itself can adjust a little for distance and direction. The style can come back later, one rule at a time, on a walk that works.
Where it stands
The kept model, tested on the 40 targets with shoves on:
- Every target reached, no falls: 6.70 s per target on average, landing 7.8–13 cm from the spot (counted as arrived within 15 cm).
- Turning round is quick: 180° on the spot in about 3.7 s; a target 1 m behind, turn and face it, in about 8.5 s.
- Upright: forward or backward lean beyond the stance 2.8° on average, never over 6.9°, against the limping model’s 8.2° average and 16.3° peak.
- Punches kept: 60 of 120, where the punching model it started from scores about 56 %.
The weak spot is stepping sideways. A sideways step covers 10–11 cm of the planned 20, so it takes more of them: 7.3–8.3 s for a target 0.8 m to the side, about as long as 1.2 m forward (7.6 s). It is also still the walk Tony criticised: it drops back towards the stance between steps, and the trailing foot swings low, heel up.
The path, in numbers
| run | what changed | outcome |
|---|---|---|
| walk1 | walk to a spot and a facing; the punching model as teacher | steps work; pivots turn 0°: the reference twisted a loaded foot |
| walk2 | pivot over a planted lead foot, 30° per beat | pivots learnt; steps stop |
| walk3 | walking rewards several times stronger | steps start landing, slowly |
| walk4 | servo feed-forward; 30 % of episodes start mid-step | steps reach 56–92 % of plan; 6/8 targets |
| walk5 | a reached spot stays reached (no circling) | 8/8 targets, no falls; punches drifting |
| walk6 | lower learning rate, tighter landing | 8/8 in 9–11 s, 5–8 cm |
| walk7 | teacher weight 2 → 10 | stopped: the teacher used the wrong input scaling |
| walk8 | teacher on its own scaling | punches back (107/200 against 112); 7.4 s per target |
| walk9 | steer through the rewards | not learnt, slower |
| walk10 | steering steps in the reference | 6.7 s per target, full-length steps |
| walk11 | 60° left pivots | 180° turn in 3.5 s; 6.3 s per target |
| walk12 | walk on through the stance | 40/40, 6.09 s; hips still slow in the stance |
| walk13 | reward the hips’ speed | no change: the reference still stopped in the stance |
| walk14 | no stance between steps; feet lifted, scraping costs | flowing walk; feet still scrape |
| walk15 | flat rear foot in the air, lifted 7 cm | shorter steps, 7.1 s |
| walk16 | lead heel down between steps | 7.4 s; sits back on the rear foot |
| walk17 | arrival within 15 cm; weight-share reward | 6.7 s, but limps and bows |
| walk18 | trunk-upright reward; weight share off | no bow; can’t step left, 32/40 |
| walk19 | reward the hips’ path | worse: 17/40 |
| walk20 | the old sideways steps back | stopped after 5 minutes for walk21 |
| walk21 | walk12’s walk, no look rewards except upright | 40/40, 6.70 s, no bow: kept |
| walk22 | allow sideways lean | shorter steps, 38/40 |
What it cost
Every training run used one c8a.24xlarge on Amazon EC2, a machine with 96 vCPUs, rented as a spot instance in us-east-1 at $1.48–1.80 an hour. When that type had no spare capacity, the launch script fell back to the same size from the same generation (c8i), and once to the generation before (c7a). It ran 33–38 M simulation steps an hour. Amazon took the machine back mid-run five times. Every run saves a checkpoint once a minute, so each time the run carried on from where it stopped on a new machine, losing a few minutes.
| item | machine-hours | cost |
|---|---|---|
| 22 training runs | about 19 | about $29 |
| about 30 test and table-building jobs on smaller machines | about 5 | about $2 |
| two speed tests of the training layout (one hung for 2 h) | about 2.4 | about $3.50 |
| total | about 26 | about $35 |
The tests ran on separate, smaller machines, so the training machine never stopped to evaluate. One of the speed tests found that splitting the training work in two ran 4 % faster, which every run since has used. These are estimates from each run’s start and stop times and the spot price at the time, not from the bill.
What’s next
The walk gets the fighter where it needs to be. It doesn’t yet look like Tony walking, and it is slow sideways. Two ideas not yet tried: plan shorter sideways steps, so the plan matches what the fighter actually does, or allow more steps per target.
The next skill is already building on this model. Blocking training started from it, with the walk, the stance and the punches all held by the same kind of teacher, so the fighter can move, punch and defend in one body.