Profiling

A program is usually slow because of one or two parts, and they are rarely the ones you would guess. A profiler runs the program and records how long each function, or on a GPU each kernel, takes, and how often it runs. The output is a list sorted by time.

Two rules make it useful:

• Speed up the biggest item first. If a part takes 10 % of the time, making it infinitely fast saves only 10 %.
• Measure again after each change. Removing one bottleneck reveals the next, and a change that should help sometimes doesn’t.

GPU work is harder to time because the CPU sends work ahead and carries on. A timer on the CPU measures sending, not doing, unless it waits for the GPU or uses the GPU’s own timers.

In this project

In Training on a rented GPU, profiling on an RTX 5090 found one kernel, the mass-matrix factorisation, taking 53 % of the GPU time for the muscular body. Replacing it made the physics 28 % faster. A later profile of the simple body found 21 ms of each step outside the physics, more than the 11 ms of physics, 8 ms of it in looking up the reference motion. Earlier, profiling a punching run on the CPU found that most of the muscular body’s contacts were its own fingers touching inside the fists (Blocking).