How webcam workout games work

Webcam workout games turn your body into the controller. No sensor, no wearable: just the camera on your phone or laptop and a pose-estimation model.

1. Pose estimation finds your joints

A pose-estimation model looks at each camera frame and returns the positions of body landmarks such as shoulders, elbows, hips and knees. Flappy Reps uses MediaPipe Pose, an open model from Google that runs in the browser fast enough for real-time play on a phone.

2. The game turns a joint into a control

The game picks the joint that moves most during the exercise and reads its height:

• Push-ups: the shoulders go up and down with every rep. Flappy Reps also uses how big your shoulders look to the camera, because on the floor your body moves toward the lens as much as up and down.
• Squats: the hips travel the furthest, so they drive the bird.
• Plank: the hips should not move at all, so the game rewards staying level.

3. Calibration makes it fair

Everyone's range of motion is different. Before a run, Flappy Reps watches one full rep, then stretches your personal range onto the whole screen. A short player doing knee push-ups and a tall athlete doing full push-ups get the same control.

4. Privacy: it all runs on your device

Because the model runs in the browser, the video does not need to go anywhere. In Flappy Reps, camera frames never leave your phone or laptop. There is no account and no upload.

Try it

Flappy Reps is a Flappy Bird style game built this way, with push-up, squat and plank modes.