Get in touch

What can be reconstructed from one helmet camera on a velodrome?

By ONMEPublished

From a 22 s excerpt of a helmet camera on a track cycling ride, structure from motion registered 182 of 184 frames with a mean reprojection error of 0.985 px, and 16,130 sparse points are shown. Metric scale comes from closing the camera path into one lap of the track's published 250 m length, about 1.5 percent either way. This is a reconstruction, not a simulation: it has no surfaces and no physics.

Camera path and points from the clip

Input clip

Points seen so far: 0

Drag to turn the view. With the keyboard, arrow keys turn it and Home resets it.

ONME computation from the footage, not ground truth. Structure from motion on every third frame of a 22 s excerpt: 182 of 184 frames registered, with a mean reprojection error of 0.985 px, and 16,130 points are shown after filtering. Metric scale comes from closing the camera path into one lap of the track's published 250 m length, about 1.5 percent either way; drift along the open path is not measured. Up is the normal of the path's plane, and the infield floor is taken as level, 1.83 m below the camera. Colour is height above that floor, 0 to 9 m. The 3D view keeps the roof points, which sit above 9 m and so show as orange. Moving riders are excluded, and the points are sparse, not a mesh.

“Indoor cycling on the velodrome of Alkmaar, Netherlands. W. Gopro hero3 black” by Patrick Savalle, via Wikimedia Commons. CC BY 3.0. Excerpt 8:28 to 8:50, resized, audio removed, advertising, stands and infield blurred; 3D reconstruction by ONME.

Research uses

  • Training in a rebuilt scene: in a robot soccer study, radiance fields reconstructed from photographs rendered the static pitch while a physics engine simulated the ball and robots, and policies trained this way transferred to physical robots (Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learning, arXiv 2405.02425).
  • Skills for a simulated player: Learning to Ball trains a simulated basketball player in a physics engine from motion capture, glove-recorded finger motion and poses extracted from internet video, without ball trajectories (Learning to Ball: Composing Policies for Long-Horizon Basketball Moves, arXiv 2509.22442).
  • Digital twins from small amounts of real data: Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulation (arXiv 2403.03949) builds them on the fly and reports a policy robustness increase of over 67 percent.

What a recording needs

  • Reconstruction, novel-view rendering and simulation are different outputs. A reconstruction says where surfaces, cameras and bodies were. Rendering shows a scene from a new viewpoint. A simulation also needs collision geometry, physical parameters and agents that can act.
  • Camera poses need metric scale from device SLAM, stereo, or structure from motion with a known dimension. The reconstruction on this page takes its scale from a known dimension, the 250 m lap of the track, and its up direction from the plane of the camera path. From one camera alone, with no known dimension, both would be unknown. Source: Sportpaleis Alkmaar, track description by its builder.
  • Coverage of a space affects reconstruction quality. Project Aria's authors compare two NeRF reconstructions, one from a careful hand-held recording that covers the space and avoids rapid motion and one from a recording made during a natural activity, and state that the difference in quality is clearly visible. Source: Project Aria: A New Tool for Egocentric Multi-Modal AI Research, arXiv 2308.13561, section 6.2.
  • Recovering camera positions from ordinary egocentric video is a processing problem in its own right: EPIC Fields reports that structure-from-motion often fails on such video, and its pipeline reconstructs 96 percent of EPIC-KITCHENS videos. Source: EPIC Fields: Marrying 3D Geometry and Video Understanding, arXiv 2306.08731.
  • Moving people and objects must be removed or modelled, as the reconstruction on this page leaves out the riders ahead, who move, and body motion in the world frame is a separate step. Physical parameters such as stiffness are a separate estimate: EgoPhys, a preprint, estimates spring stiffness for deformable objects from egocentric video with priors learned from many interactions (EgoPhys: Learning Generalizable Physics Models of Deformable Objects from Egocentric Video, arXiv 2606.16202).
  • A simulation is validated against held-out real observations or real deployment, as the robot soccer and real-to-sim-to-real studies do. A rendered scene alone is not physics-ready.

Where the format matters in sport

Where the format matters in sport
Track cyclingA velodrome has a known size, so one lap sets the scale: the reconstruction on this page rests on the 250 m lap of the Alkmaar track and covers a 97 by 48 m oval, with riders on the track left out because they move. Source: Sportpaleis Alkmaar, track description by its builder.
BoulderingDepth helped in one benchmark: on vertical walls off-the-shelf video methods placed climbers about 1.4 to 1.5 m off in world coordinates, and a method that adds LiDAR depth to RGB reached about 0.1 m. Source: ClimbingCap: Multi-Modal Dataset and Method for Rock Climbing in World Coordinate, arXiv 2503.21268.
BasketballLearning to Ball's authors attribute the remaining gap between a simulated basketball player and skilled players mainly to limited reference data. Reconstructed motion is one possible source of reference data (our reading). Source: Learning to Ball: Composing Policies for Long-Horizon Basketball Moves, arXiv 2509.22442.
FootballThe pitch in the robot soccer study linked below was rebuilt as radiance fields from 250 to 300 photographs. Source: Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learning, arXiv 2405.02425.

Limits

This is a reconstruction, not a simulation: it has no surfaces and no physics, and the points are sparse, not a mesh. Riders ahead are left out because they move.

Drift along the open path is not measured. Metric scale rests on closing the path into one lap of the track's published length, about 1.5 percent either way, and the result is an ONME computation from the footage, not ground truth.

Sources

  1. Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learning, arXiv 2405.02425Read
  2. Learning to Ball: Composing Policies for Long-Horizon Basketball Moves, arXiv 2509.22442Read
  3. Reconciling Reality through Simulation, arXiv 2403.03949Read
  4. Project Aria: A New Tool for Egocentric Multi-Modal AI Research, arXiv 2308.13561Read
  5. EPIC Fields: Marrying 3D Geometry and Video Understanding, arXiv 2306.08731Read
  6. EgoPhys: Learning Generalizable Physics Models of Deformable Objects from Egocentric Video, arXiv 2606.16202Read
  7. ClimbingCap: Multi-Modal Dataset and Method for Rock Climbing in World Coordinate, arXiv 2503.21268Read
  8. Sportpaleis Alkmaar, track description by its builderRead