Camera calibration

A photo flattens the world. To go back from a pixel to a direction in space you need two kinds of numbers about the camera:

• Intrinsics: the lens. Its focal length $f$ (in pixels), the image centre $(c_x, c_y)$ and how much the lens bends straight lines (distortion).
• Extrinsics: where the camera is and which way it points, a rotation $R$ and a position $t$ in the room.

Together they say where a 3D point $X$ lands in the picture:

$$\lambda \begin{pmatrix} u \\ v \\ 1 \end{pmatrix} = \begin{pmatrix} f & 0 & c_x \\ 0 & f & c_y \\ 0 & 0 & 1 \end{pmatrix} \big( R X + t \big)$$

where $(u, v)$ is the pixel and $\lambda$ the point’s depth in front of the camera.

Calibrating means finding these numbers. The usual way is a printed pattern of known size, such as a checkerboard. Its corners are easy to find in a picture to a fraction of a pixel, and since their true spacing is known, the camera’s numbers can be solved for. A ChArUco board adds small printed codes to the squares, so each corner is identified even when part of the board is hidden.

In this project

The coach’s motion capture uses two phones. A ChArUco board (7 × 5 squares) lies flat on the floor and each phone takes one still photo of it from where it will film; then the board is taken away and the phones record without moving. A 12-megapixel still reads all 24 inner corners from 2.4 m where a video frame can’t. The lens comes from the photo’s focal length, and the video camera from matching the photo to the room. In the first real session the two phones came out 89° apart, and the floor fitted to his planted feet was within 2.6° of the board’s.