The jab: the simplest punch, and a long lesson

The jab is the first punch anyone learns. Lead hand out, lead hand back. Tony Yum has taught it to beginners for years, so he expected it to be the easy one for the AI fighter too.

It wasn’t. Over two days and 23 training runs, the jab turned out to hold most of the lessons of the project so far. This post follows that path from the coach’s side. The previous posts cover how the fighter learnt to stand and how the coaching works.

What a jab really is

To a beginner, a jab is an arm movement. To a coach, the arm is the last link of a kinetic chain that starts at the feet. Tony’s spec for the reference motion read like a checklist he would give a student:

• The lead foot steps forward a little (16 cm) as the fist goes out, and turns with the hips.
• The hips turn about 45° to the right, so the left shoulder goes forward and the right shoulder back.
• The fist turns over on the way, so the palm faces the floor at full extension.
• The weight stays about 50/50 between the feet. The jab doesn’t fall onto the front leg.
• The chin stays tucked behind the lead shoulder. The rear glove comes up beside the right cheek.
• After the snap the fist comes straight back to the guard, and the lead foot steps back into the stance.

The fist goes out in 0.14 seconds and is back in the guard 0.14 seconds later. The video at the top of this post shows the result.

There is a catch. The reference is an animation. It was built pose by pose with inverse kinematics, shaped by Tony’s corrections until it looked right. Nothing in it obeys physics. It has no mass, no momentum and no feet that slip. It is, in Tony’s words, vibe-created. Figure 1 shows its key moments, with a line down from its centre of mass to show where the weight sits.

The fighter then has to do the same movement in a physics simulation, where everything has weight. A reward function scores how well it does. The policy tries millions of jabs and keeps whatever scores best. So everything below is about one question: when the animation and physics disagree, which parts of the animation does the coach insist on, and which does he let go?

Lesson 1: reaching out and touching

The first run copied the reference and paid a reward for hitting a target at full reach. The fighter kept its balance and stretched its arm out, but never stepped. Every jab landed 15 cm short.

The reason was simple arithmetic. A hit paid about 2 % of what a whole episode paid. Standing still in a good stance paid the rest. So the safe choice was to keep the feet planted. With the hit reward made four times bigger, the fighter started to step and hit. After target distances were added as a curriculum, it hit within 5 cm on 93 % of full-reach jabs.

Tony watched the videos and wasn’t impressed. “It’s reaching out and touching.” There was no speed and no power.

This was the first clash between the animation and physics. The reference fist slowed to zero on the target and held there for 0.04 seconds, like a finger pressing a doorbell. The reward paid for every moment the fist stayed near the target, so arriving gently and staying there scored best. The policy did exactly what it was paid for, a mild case of reward hacking.

So the reference changed: no hold, and a quicker jab. The target moved 7 cm inside the point where the reference’s fist stops, so the punch goes through it, as a real jab goes through the target. The reward changed too. One payment per jab, multiplied by the fist’s speed when it crosses the target.

The fist’s top speed rose from 5.5 to 7.4 m/s. Jabs at nearer targets now hit at 4.4 to 5.4 m/s, against about 1 m/s before.

Lesson 2: the step that never came back

Then Tony saw something in the grid of test videos. After each jab, the fighter stayed where it had stepped. A second later it faced 29 to 39° to the right, with both feet 10 to 20 cm from where they had started.

Every part of the stance reward was measured in the body’s own frame. A stance a little forward and turned to the side scored just as well as the original one. The fighter had no reason to go back, and no way to see where it had started.

The obvious fix was a “home” reward: after each punch, face forward again with the feet back in place. The fighter learnt that. But it stopped stepping into the jab at all. Its hips ended up 15 to 18 cm behind the reference at impact, and the punch lost its power.

Was the home reward stopping the step? Two runs went side by side to find out, one with the home reward and one without. Neither stepped. Both lost the jab’s speed, from 1.5 m/s at the target down to 0.4 to 0.7 m/s. Meanwhile their total scores went up, from the stance terms. Not stepping had become a local optimum. Stepping in and back would have scored more. But it is a long, coordinated sequence the policy had never found, and every small step towards it scored worse at first.

Here Tony made a coaching call:

The speed is important and so is the style before and on impact. But the step back can be more relaxed stylistically, to make it easier to recover.

So the reward split the jab in two. Up to the impact it asked for style and speed: the reference’s turn, the foot placement, and a fist moving at least as fast as the reference’s. After the impact, the stance reward eased off to simply “stay up and keep your guard” for half a second. The home target also got twice as forgiving, 20° and 12 cm.

It worked straight away. The new run hit on 54 % of its training jabs at 4.4 m/s. The old reward, over the same time, managed about 0 % at 0.6 to 1.2 m/s.

Lesson 3: the hips, and how much rotation to ask for

The arm was now fast, but the hips stayed 11 cm behind the reference at impact. That is an arm punch, the beginner’s mistake of jabbing without the body behind it. In the reward, the hips were worth about 3 points per jab against about 45 for the fist’s speed and the hit. The policy had worked out that the hips didn’t matter much.

The next run multiplied the impact payment by how close the hips were to the reference’s at impact. Hits dropped to 17 % at first, because the impact suddenly paid less. Then they recovered. By the end the hips were 6 cm behind, with 57 % hits at 4.0 m/s. Most of the gap had been the step’s length, not the hips’ turn.

Tony then gave the body some freedom:

If the hip movement, that rotation, the hip and the feet, is costing the jab speed and accuracy, focus on the top half of the body. The step is still important, stepping forward is important, but the actual rotation can be relaxed a bit.

So the hip score now counted only how far the hips lagged forward, not their sideways offset or their turn. The turn’s tolerance widened from 8° to 20°. A new term asked the arms and shoulders to follow the reference up to the impact. The jab improved again: 66 % hits at 4.5 m/s, hips 5.7 cm behind.

Lesson 4: the bow

Freedom has a price. Tony spotted it in the next grid:

After the jab and the recovery, the character wobbles forward like a little bow.

Half a second after each jab, the chest pitched 13 to 16° forward, then straightened up. The relaxed return asked only “stay up and keep your guard”, and a bow does both. The body had found the cheapest way to soak up the punch’s momentum.

The fix was to give the trunk back one rule during the relaxed return: keep the chest within 4° of the reference’s tilt. The legs kept their freedom. The bow went from about 15° to under 4°. The jab got better too: 76 % hits at 4.6 m/s, and the hips 4.9 cm behind, inside the 5 cm goal for the first time. Figure 2 shows one of its jabs, now under physics.

It had a side effect. With the chest held only after the punch, the fighter’s ready stance drifted back to a 5° lean. So the chest rule was extended to cover every moment except the punch’s drive itself. The stance stood upright again, with no bow, and the fighter hit 9 out of 9 test targets at every aim, up to 15 cm off centre.

Lesson 5: distance

A real opponent isn’t always at the perfect distance, so the targets moved nearer and farther. Two more coaching calls came out of it.

• Too close. A target 25 cm nearer forced a cramped jab: a bent arm and no step. Tony’s verdict: it “looks really bad”, and at that range a fighter should throw an elbow or a knee anyway. The close jab was dropped.
• Too far. For targets beyond full reach, the hips stayed behind again, 9 to 10 cm, and the jab slowed down. A reward for the hips’ position at the moment the fist crosses the target helped full-reach jabs, from about 4 to about 5 m/s. The far jab still didn’t get fast.

That run gave the most accurate jab so far: 85 % hits on a fixed test of 54 jabs, at a median of 4.7 m/s. Figure 3 is the grid Tony got on his phone from that version. The near and mid jabs hit hard. At full reach they still hit, a little slower. The far jabs get there, but slowly. The far jab is waiting for a new reference. Tony will film himself throwing a normal jab and a long one, side on, so the longer step can be copied from a real body.

The path, in numbers

The same numbers as a table:

run	what changed	hits	fist speed	hips behind at impact
home reward	come back to the stance after each jab	~0 %	0.6–1.2 m/s	12–14 cm
style	style and speed up to impact, relaxed return	54 %	4.4 m/s	10.9 cm
hips	impact pays less with the hips behind	57 %	4.0 m/s	6 cm
upper body	arms first, hip turn relaxed	66 %	4.5 m/s	5.7 cm
upright	no bow in the return	76 %	4.6 m/s	4.9 cm

Hits and speeds are from training, with the policy’s exploration noise on. Each run continued from the one before (fine-tuning), so the skill built up.

One machine quirk is worth knowing. In several later runs the training numbers held steady while the noise-free, deterministic jab slowly got worse after about 10 M steps. So runs now save a checkpoint every couple of million steps, and the best one is picked by testing the jab itself.

What it cost, and which machine

The fighter trains in the cloud on Amazon EC2, as described in the balance post. A physics simulator like MuJoCo runs on the processor, one copy of the fighter per core, so the number of vCPUs is what buys speed. A graphics card adds very little.

The default machine is a c8a.16xlarge: 64 vCPUs, about 29 M simulation steps an hour. Rented as a spot instance, from Amazon’s spare capacity, it cost about $0.97 an hour in us-east-1 over these two days. The same machine on demand costs $3.28. On that machine a typical jab run of 15 M steps took 40 to 50 minutes, so each experiment cost well under a dollar.

Three decisions shaped the bill:

• Spot, and a fallback when spot fails. Amazon can take a spot machine back at two minutes’ notice. Training saves a checkpoint every minute and a watcher relaunches the run, so a reclaim costs a few minutes. On the afternoon of 16 October one run was reclaimed six times in 90 minutes. It moved to an on-demand machine for its last 35 minutes, about $2, to finish. When no c8a was free, a c7a.16xlarge filled in: about 30 % slower, but from a different pool of spare machines.
• Two smaller machines for a side-by-side test. The account could run 64 spot vCPUs at a time. For the home-reward test, two runs had to start together with the same settings. So they each ran on a c8a.8xlarge: half the cores, at about $0.68 an hour. That is about 40 % more per core, but it gave a fair comparison in the same half hour.
• Short runs, often stopped early. Each run had a deadline and a step budget, and the grids of test videos showed problems within 5 M steps. When a run clearly wasn’t going anywhere, it was stopped and the next idea started.

Adding up the start and stop times of the 23 jab and punch runs from 16 and 17 October gives about 16 machine-hours. At the spot prices of those days that comes to roughly $16, including the on-demand finish. Two days of a coach’s ideas, tested, for the price of a lunch.

What the coach learnt

Each fix moved the problem somewhere else.

• Pay for a hit, and the fighter reaches and touches.
• Pay for speed, and it steps in and never comes back.
• Make it come back, and it stops stepping, and the jab goes weak.
• Relax the way back, and it bows.
• Hold the chest after the punch, and the stance leans back.

None of this is a bug in the AI. Each time, the policy did exactly what the reward asked. The reference was made without physics, so it can’t all be copied. The coach has to choose which details are the jab and which are decoration.

For Tony the answer came out in an order he recognises from the gym. The speed and the shape up to the impact come first. The step forward comes next, because that is where the power comes from. The chest stays upright, and the head stays up. The hip turn and the legs on the way back can be looser, as long as the fighter ends up back in its stance, guard up.

It is also the order in which he fixes a beginner’s jab. A student told to “get back to your guard” often stops stepping in. A student told to “turn your hips” sometimes turns everything and loses the target. Coaching means asking for one thing at a time, and noticing what the student gives up to get it. The fighter just does it faster, and on a phone screen.