Get in touch

How far can a worn stereo pair measure depth?

By ONMEPublished Updated

On a path surface, a cyclist's helmet cameras, 109.5 mm apart, had a matching spread of 0.35 to 0.70 px (robust 1 sigma). That implies a depth spread of 3.5 cm at 2.8 m, 12 cm at 4.8 m and 2.3 m at 16.4 m. These are spreads about a fitted path plane, not errors against measured depth: the ride has no ground-truth depth.

The recording

TUM-VIE (arXiv 2108.07329) records sequences with a rig of two global-shutter cameras, each a 1024 x 1024 px greyscale image at 20 Hz. A board triggers both in hardware, so left and right frames carry one timestamp and one exposure time. In the bike sequences the rig is on a helmet worn by the rider. The published calibration puts the cameras 109.5 mm apart, where the paper gives 10.86 cm. They are nearly parallel, with 0.50 degrees of relative rotation, but their lenses are fisheye, so the pair has to be rectified first.

We used a 12 s window (158.0 to 170.0 s) of sequence bike-easy: a daytime ride at 17 to 22 km/h along campus paths, past a railing, lamp posts, trees and a building facade. The cameras look ahead, so the bicycle and the rider are out of view. It is public research footage, not ONME data.

After rectification, a feature sits on one row

Left, rectifiedLeft, rectified
Right, rectifiedRight, rectified
Figure 1. ONME computation from TUM-VIE, not ground truth. Left and right frames after rectification with the published calibration, at 4.0 s of the clip. The 36 drawn matches are a random draw from all 480 positive-disparity matches in this frame, with no row filter; 23 of the 480 are 2 px or more off the row. Across 24 frames, 14,810 matches sit a median 0.31 px from the same image row, and 87 percent are within 1 px.

“TUM-VIE: The TUM Stereo Visual-Inertial Event Dataset, sequence bike-easy” by Simon Klenk, Jason Chui, Nikolaus Demmel and Daniel Cremers, via the TUM-VIE dataset page. CC BY 4.0. Trimmed, rectified and re-encoded by ONME. The raw view is trimmed, resized and re-encoded only. No endorsement by the dataset's creators is implied.

From shift to distance

Left, rectifiedLeft, rectified
Disparity (OpenCV SGBM)Disparity (OpenCV SGBM)facade 19.2 mlamp post 4.7 mrailing 8.0 mrailing 2.6 m
Figure 2. ONME computation from TUM-VIE, not ground truth. Disparity in pixels, found by block matching, converted to metres through the published calibration. The four labelled read-outs, a railing at 2.6 m, a lamp post at 4.7 m, a railing at 8.0 m and a building facade at 19.2 m, each agree with an independent block search within 0.6 px. Metric scale rests on the published calibration: a baseline of 109.5 mm (the dataset's paper gives 10.86 cm) and a rectified focal length of 700 px. The camera sits 2.04 m above the path, which is plausible for a helmet.

Four read-outs from that frame

Four read-outs from that frame
PointDisparity (px)Depth (m)Block search (px)
Railing, left29.02.6428.9
Lamp post16.44.6617.0
Railing, right9.67.969.5
Building facade4.0019.164.00
Disparity is the StereoSGBM median over a 9 x 9 px patch; depth is f B / d with f B = 76.6 px m. The block search is an independent normalised cross-correlation search on the same row (15 x 15 px patch). Each pair agrees within 0.6 px.

Depth spread on the path by distance

  • Model lines: matching spread of 0.25, 0.5, 1 px, Z² · Δd / (B · f) with B = 0.10949 m and f = 700 px.
  • Shaded: the matching spread measured on the surface, 0.35 to 0.7 px.
  • Dots: 6 distance bins of the surface, each the measured matching spread converted to depth.
Figure 3. ONME computation from TUM-VIE, not ground truth. Spread of depth about a fitted path plane against distance. The measured matching spread is 0.35 to 0.70 px; through Z squared times that spread divided by focal length and baseline it gives 3.5 cm at 2.8 m, 12 cm at 4.8 m, 0.8 m at 10.9 m and 2.3 m at 16.4 m. The dots follow the lines by construction, so this is not an independent test of the model.
Show the numbers
Distance (m)Matching spread (px)Depth spread (cm)Points
2.830.353.5429,774
3.450.396.01,629,515
4.790.3911.81,675,343
7.140.4530.51,021,975
10.920.5080.5642,772
16.360.70226.6311,125

What the spread implies

A fixed matching spread in pixels is a larger share of a smaller disparity, so depth uncertainty grows with the square of distance. At the measured 0.50 px at 10.9 m, where the disparity is about 7 px, the standard stereo model gives a relative error of about 7 percent. At 16.4 m the disparity is about 4.7 px and the same model gives about 14 percent. One pixel of disparity is about 5 m of depth at 20 m, so beyond about 20 m depth is coarse.

For the pixels that passed the checks, the median depth was 10.2 m. Sky and smooth asphalt were the main losses.

Method

We rectified both views with the published Kannala-Brandt calibration into a virtual pinhole pair (1024 x 1024 px, f = 700 px, a 72.4 degree horizontal field). To check that the calibration still fits this sequence, we matched SIFT features on 24 frames; the first figure gives the result.

Disparity came from OpenCV StereoSGBM. A pixel kept its value only if the left-to-right and right-to-left searches agreed within 1 px, both views saw it, and the mean horizontal gradient over a 7 x 7 window was at least 4.5 grey levels; without that texture check, smooth walls are filled with values that cannot be verified.

For the spread, we fitted the path plane in disparity space by RANSAC on the part of the image below its centre, in 24 frames, and took the robust spread of the residuals, binned by the plane's depth.

Where depth is missing

LeftLeft
RightRight
DisparityDisparity
Figure 4. ONME computation from TUM-VIE, not ground truth. Open sky and smooth walls give nothing to match, a thin pole is widened or broken up, and beyond about 20 m one pixel of disparity is about 5 m of depth. A median of 53 percent of the left view gets a disparity per frame (39 to 67 percent) after the left-right, shared-view and texture checks.

Speed from the pair

A stereo pair fixes metric scale, so camera motion between frames is a length. We matched stereo points to the left image 0.25 s later and solved the camera's motion with PnP and RANSAC, every 2.5 s. Over the cycling part of the ride (102.5 to 197.5 s) the median speed was 19.3 km/h, with a 10th to 90th percentile of 15.8 to 24.0 km/h, and 17 to 22 km/h in the clip. The estimate includes the head's own motion, and no speedometer or GPS checks it.

Research uses

  • Dense depth from rectified pairs: the Aria Gen 2 Pilot Dataset (arXiv 2510.16134) computes depth images from the glasses' front camera pair with a stereo model, and reports errors that increase with distance and vary from frame to frame.
  • Zero-shot stereo matching, a model class that estimates dense depth from rectified pairs without per-domain fine-tuning: FoundationStereo: Zero-Shot Stereo Matching (arXiv 2501.09898).
  • Wearer body pose from stereo views: 3D Human Pose Perception from Egocentric Stereo Videos (arXiv 2401.00889) uses depth features from a 3D scene reconstruction module.
  • Hand and object tracking: HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videos (arXiv 2411.19167) finds that multi-view methods significantly outperform single-view ones. Its views are several cameras on one headset, the stereo arrangement extended to more cameras.

What a recording needs

  • Stereo is two cameras on one wearer with a fixed, calibrated baseline. Several wearers, whose distance to each other changes over time, are multi-egocentric video instead.
  • Depth is inversely proportional to the shift between the two images: Z = B f / d for baseline B, focal length f and disparity d. Both images are taken at the same moment, so this also works for moving people and balls, and it fixes metric scale. Source: OpenCV depth map tutorial.
  • On Aria Gen 2 the two front computer-vision cameras form a stereo pair with significant overlap; its single colour camera is separate. Project Aria's Gen 2 announcement reports a stereo overlap of 80 degrees, up from 35 degrees on Aria Gen 1. Source: Aria Gen 2 hardware specification.
  • Calibration of the pair must be kept current: in one recording, taking off first-generation Aria glasses rotated the two cameras by about 25 arcmin, roughly 1.5 px. Source: Project Aria: A New Tool for Egocentric Multi-Modal AI Research, arXiv 2308.13561, section 4.2.
  • Rectification puts matches on the same image row. Matching timestamps are not enough if the shutters fire at different times; a global shutter avoids row-by-row skew during head motion.
  • Precision falls with distance, as the depth spread figure shows. Only the region seen by both cameras has depth, and plain surfaces give nothing to match. Source: TUM-VIE dataset page.

Where the format matters in sport

Where the format matters in sport
CyclingA helmet pair gives a rider's speed without a sensor on the bike; the estimate, with its median, spread and limits, is under Speed from the pair. Source: TUM-VIE dataset page.
BadmintonA shuttle is fast, so both cameras must expose at the same instant: at 89.6 m/s it moves about 4.5 m between frames at 20 fps, the rate of the ride clip analysed in this note (ONME arithmetic). A legged robot that returns shots uses a body-mounted stereo camera (arXiv 2505.22974), a robot's and not a wearer's. Source: A Correlational Analysis of Shuttlecock Speed Kinematic Determinants in the Badminton Jump Smash.
BoulderingIn one benchmark, off-the-shelf video methods placed climbers about 1.4 to 1.5 m off on vertical walls, and a method adding LiDAR depth to RGB reached about 0.1 m. Whether worn stereo depth gives a similar gain is untested there. Source: ClimbingCap: Multi-Modal Dataset and Method for Rock Climbing in World Coordinate, arXiv 2503.21268.

Limits

No ground-truth depth or speed exists for this ride, so none of the figures above is an error against measured values; motion capture covers only the indoor start and end. Metric scale rests on the published calibration: the paper and the calibration file differ by 0.8 percent on the baseline, and the path plane gives a plausible camera height of 2.04 m, which is a plausibility check and not a scale measurement.

The spread is measured on an asphalt path and includes surface texture, so it bounds the matching noise from above for that surface. Frames are 8-bit JPEGs from the dataset archive, not the cameras' 12-bit output. The two sensors share one timestamp per frame, which fits hardware triggering; the start of each exposure was not independently measured.

This is one helmet rig on one daytime ride, in greyscale. It says what a calibrated pair on a head can do, not what any other camera does.

Sources

  1. TUM-VIE dataset page and calibration fileRead
  2. TUM-VIE: The TUM Stereo Visual-Inertial Event Dataset, arXiv 2108.07329Read
  3. OpenCV 4.13.0 tutorial: depth map from stereo imagesRead
  4. Aria Gen 2 Pilot Dataset, arXiv 2510.16134Read
  5. FoundationStereo: Zero-Shot Stereo Matching, arXiv 2501.09898Read
  6. 3D Human Pose Perception from Egocentric Stereo Videos, arXiv 2401.00889Read
  7. HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videos, arXiv 2411.19167Read
  8. Aria Gen 2 hardware specificationRead
  9. Aria Gen 2 announcementRead
  10. Project Aria: A New Tool for Egocentric Multi-Modal AI Research, arXiv 2308.13561Read
  11. A Correlational Analysis of Shuttlecock Speed Kinematic Determinants in the Badminton Jump SmashRead
  12. Learning coordinated badminton skills for legged manipulators, arXiv 2505.22974Read
  13. ClimbingCap: Multi-Modal Dataset and Method for Rock Climbing in World Coordinate, arXiv 2503.21268Read