Blog
Technical write-ups from training a simulated martial artist.
Training on a rented GPU: four and a half times cheaper
Spot CPU machines on EC2 were getting expensive. Moving the fighter's physics onto a GPU, tuning the code for it and trying nine graphics cards cut the cost of training the stance by about 4.5 times.
Blocking: reading a punch you see a tenth of a second late
Twenty runs taught the fighter four Muay Thai blocks, then to watch an opponent and pick the right one. Four times it found a giveaway in the opponent instead of reading the punch.
Footwork: walking to a spot in a Muay Thai stance
Twenty-two runs taught the fighter to step forward, back, sideways and diagonally, and to pivot, without forgetting its punches. Then every rule about how the walk should look made it walk worse.
Faster punches: from trained amateur towards professional
The five punches landed, but at an ordinary person's pace. Seven runs, a policy that sets its own timing, and a stronger elbow brought every punch to the target sooner, and finally landed the cross.
Five punches: the cross, the hooks and the uppercut
Adding four punches to the jab, then air, pads and a heavy bag: ten training runs, a coach's corrections, and a fighter that now throws a near-human combination.
The jab: the simplest punch, and a long lesson
Teaching a simulated fighter to jab: why a punch copied from an animation doesn't survive physics, and how each fix moved the problem somewhere else.
Coaching an AI fighter
A Muay Thai coach on teaching a simulated fighter to punch: why it is so much like teaching a person, and how he does it from his phone.
Learning to balance in a Muay Thai stance
How a simulated humanoid learned a coach-specified Muay Thai stance, then learned to take shoves of up to 860 N without falling and with its guard up.