Get in touch

How reliably does a pose model find a rider from a fixed camera?

By ONMEPublished

On a 10 s tripod clip of a rider and hardtail bike coming down rock ledges, a pose-landmark model found a rider in 299 of 300 frames, the only miss at frame 0, with a median of 28 of the 33 landmarks at 0.5 visibility or more. Lower-body points are often missed: from 3.7 to 4.7 s the bike hides the legs and they are mislocated.

The clip

A tripod camera sees the whole rider and bike come down rock ledges: body position, bike path and ground in one frame, which a camera on the rider's helmet cannot show. Where the rider looked is not recorded.

Frames in which a pose model found a body

Loading detections.

Model estimate (MediaPipe), not ground truth. A strip of the clip's 300 frames, marked where a pose-landmark model found a rider, with the skeleton drawn on the video: 299 frames, the only miss at frame 0. A median of 28 of the 33 landmarks per detected frame reach 0.5 visibility. From 3.7 to 4.7 s the bike hides the legs and the lower-body points are mislocated. The model's documentation puts more than one person, people further than about 4 m and a head that is not visible outside its scope. Only 2D image positions are kept.

“Technical Hardtail riding [ all mountain / enduro ]” by Radde faehrt Radd (YouTube channel), via Wikimedia Commons. CC BY 3.0. Trimmed to 10 s and re-encoded without audio by ONME.

Showing the video.

Where the legs are hidden

The left ankle is below the 0.5 visibility threshold in 215 of the frames with a pose and the right in 126, so lower-body points are often not drawn.

Research uses

  • Action spotting in full broadcast matches: SoccerNet: A Scalable Dataset for Action Spotting in Soccer Videos (arXiv 1804.04527) labels goals, cards and substitutions in 500 complete games, anchored at one-second resolution, about one event every 6.9 minutes.
  • Tracking several players: SportsMOT: A Large Multi-Object Tracking Dataset in Multiple Sports Scenes (arXiv 2304.05170) holds 240 sequences of basketball, volleyball and football, where the key challenge is object association.
  • Multi-view 3D body pose and mesh under occlusion: Harmony4D: A Video Dataset for In-The-Wild Close Human Interactions (arXiv 2410.20294) uses more than 20 synchronised cameras over 208 sequences of wrestling, dance, karate, MMA and fencing.

What a recording needs

  • For 3D tracks or poses, each camera needs intrinsic and extrinsic calibration. Harmony4D determines all camera parameters with structure from motion and scales its world to be metric and gravity-aligned, after a pre-scan of the environment.
  • Several cameras give one set of tracks and poses only if their files are synchronised and calibrated, with the timing error measured.
  • Hands and small objects are small and far in an external view, one camera can hide them behind the body, and the view does not record what a participant looked at.

Where the format matters in sport

Where the format matters in sport
FootballSoccerNet labels 500 broadcast games with goals, cards and substitutions anchored to one-second resolution. A broadcast camera follows the ball; recording every player and the space between them needs fixed wide or tactical cameras. Source: SoccerNet: A Scalable Dataset for Action Spotting in Soccer Videos, arXiv 1804.04527.
Distance runningAn external camera can measure gait: the study linked below used one side-on camera sampling at 240 Hz, about 6 m from the runners. Source: Reliability of Overground Running Measures from 2D Video Analyses in a Field Environment, PMC6359244.
BoulderingA wall-facing external camera sees all four limbs but not always the hands, so camera placement matters. From one handheld camera facing the wall, a hand on a hold in The Way Up was more than half covered by the climber's body for 50.53 percent of its hold-use time on average, a foot for 2.19 percent. Source: The Way Up: A Dataset for Hold Usage Detection in Sport Climbing, arXiv 2505.12854.

Limits

The pose is a model estimate, not ground truth, and its scope is the one stated in the model's documentation (MediaPipe pose landmarker model card and scope). A single fixed camera can also hide a limb behind the body or the bike, as it hid the legs from 3.7 to 4.7 s.

Sources

  1. SoccerNet: A Scalable Dataset for Action Spotting in Soccer Videos, arXiv 1804.04527Read
  2. SportsMOT: A Large Multi-Object Tracking Dataset in Multiple Sports Scenes, arXiv 2304.05170Read
  3. Harmony4D: A Video Dataset for In-The-Wild Close Human Interactions, arXiv 2410.20294Read
  4. The Way Up: A Dataset for Hold Usage Detection in Sport Climbing, arXiv 2505.12854Read
  5. Reliability of Overground Running Measures from 2D Video Analyses in a Field Environment, PMC6359244Read
  6. MediaPipe pose landmarker model card and scopeRead