Triangulation

One camera can’t measure depth: every point along a ray from the lens lands on the same pixel. A second camera, looking from somewhere else, gives a second ray. Where the two rays meet is the point.

Real rays never meet exactly (the pixels are a little off), so the point is chosen to fit both as well as possible. With each camera’s projection matrix $P_i$ from calibration and the point seen at pixel $(u_i, v_i)$, each camera gives two linear equations in the unknown point $X$:

$$u_i \, (P_i^{3} X) - P_i^{1} X = 0, \qquad v_i \, (P_i^{3} X) - P_i^{2} X = 0$$

where $P_i^{k}$ is row $k$ of $P_i$. Stacking them for every camera and solving by least squares gives $X$.

Depth is measured best when the cameras are far apart in angle: two cameras side by side see nearly the same ray, and a pixel of error becomes a large error in depth. 60–90° apart is a good spread for a person.

In this project

The coach’s motion capture triangulates the 2D keypoints from two phones about 89° apart, one where the opponent stands and one on the side of the striking limb. An earlier attempt with one phone had to guess every limb’s depth from its length, and once put the legs the wrong way round in the stance while every picture check passed. With two views, which foot is in front is measured, not guessed.