Checkpoint

A checkpoint stores everything needed to carry on training exactly where it left off:

• the network weights;
• the optimiser’s state (for Adam, its running averages);
• the observation normalisation statistics;
• the training position: step count, curriculum level, random seed.

Saving only the weights is enough to use a policy, but not to resume training smoothly.

Why save often

• Interruptions. A spot instance can be taken away with two minutes’ notice. Without a recent checkpoint, the work since the last save is lost.
• Collapses. If training goes wrong (see training divergence), an earlier checkpoint is a working fallback.
• Comparison. Checkpoints from different points let you test and film the policy as it improves.

A checkpoint chosen as the starting point for later work is often called frozen: it’s kept unchanged, and new runs start from a copy.

In this project

Training saves a checkpoint to Amazon S3 every 60 s. If the cloud machine is reclaimed, a watcher relaunches the run on a new one and it resumes from the latest checkpoint. Good policies are copied to a frozen/ folder, like the stance-and-rhythm policy that both shove-recovery approaches started from.