Sensor synchronisation and calibration specification¶
What this project assumes about time and calibration across its sensor streams, what it has measured on real recordings, and what a system built on it would still have to supply. It is the item listed as missing under Track C of the roadmap (P5-07). It specifies the evaluation harness, not a product: there are no sensor drivers here (see "Deliberately not planned").
1. Streams and the timing model¶
| Stream | Source in the harness | Rate | Timestamp |
|---|---|---|---|
| IMU | Synthetic generator, or a recording (EuRoC, TUM VI) | 200 Hz nominal | Seconds from the first IMU sample; nanoseconds on disk |
| GNSS position | Generated from the reference trajectory, never recorded | 5 Hz by default | Seconds, on the reference clock |
| Visual relative pose | A surrogate generated from the reference, never images | 20 Hz by default | Seconds, on the reference clock |
| Ground truth | The generator's analytic path, or a recording's reference | 120 to 200 Hz in the recordings | Seconds, on the recording's clock |
- One clock. Every stream is on one time axis. A recording's timestamps are read as integer nanoseconds and converted to seconds after the first IMU sample is subtracted. A float64 second holds about 0.24 microseconds of a nanosecond clock at the magnitudes these datasets use, which is the floor of the jitter figures below.
- Association. The filter integrates the IMU at its own timestamps and applies a GNSS or visual measurement at
the first IMU sample whose time is at or after the measurement's. The measurement is applied to the state at
that sample, not propagated or interpolated back to its own time. The delay is under one IMU interval and is
reported as
max_measurement_latency_s. At the speeds of these datasets it is a few millimetres. - No latency compensation, no online offset estimation. The filter has no time-offset state and does not
estimate a delay between streams. An offset between two streams is injected as a scenario setting
(
gnss_time_offset_s,vision_time_offset_s,imu_time_offset_s) and is not corrected. - Rates are taken from the timestamps. The IMU is not assumed uniform: the filter uses the spacing the timestamps give.
2. What time offsets do, as measured on the synthetic fixture¶
The fault matrix injects GNSS timestamp offsets and vision timestamp offsets. A GNSS offset degrades calibration and is not detected; the effect grows with the offset. A vision offset shows nothing on a case that is already overconfident. An IMU offset is not measured. The stochastic-clone cases have not been run with a vision offset. These are statements about the synthetic path and say nothing about a real platform's speed.
3. What the recordings show¶
The recorded IMU, ground truth and clock were measured, not assumed. The table is generated by
navkit euroc timing from data/timing.csv; a test fails if they disagree. CI cannot
regenerate it, because the datasets are not vendored.
How to read it. Longest gap is the largest spacing between consecutive timestamps. Truth time in dropouts is the share of the recording that lies inside ground-truth gaps longer than 0.1 s. Offset is the shift of the ground-truth timestamps that best matches the IMU, found by matching the angular-rate vector of the ground-truth attitude against the gyroscope axis by axis after removing each axis's mean, which removes a constant gyroscope bias; it assumes the ground truth is expressed in the IMU frame, as both datasets state. A positive value means the ground-truth timestamps are early. Magnitude check is the same offset found by matching only the size of the rate, which needs no frame but is shifted by a few milliseconds by a gyroscope bias, so it checks the order of magnitude and is not the figure. Correlation is the correlation of the two rate series at the best shift and at zero shift. Offset x speed is the displacement a shift of that size causes at the sequence's median speed.
| Sequence | IMU Hz | IMU longest gap ms | Truth Hz | Truth longest gap ms | Truth time in dropouts | Offset ms | Magnitude check ms | Correlation (peak / at zero) | Offset x speed mm |
|---|---|---|---|---|---|---|---|---|---|
| MH_01_easy | 200.0 | 5.0 | 200.0 | 5 | 0.0% | +0.0 | -0.1 | 0.989 / 0.989 | 0.0 |
| MH_02_easy | 200.0 | 5.0 | 200.0 | 5 | 0.0% | -0.0 | -0.1 | 0.989 / 0.989 | 0.0 |
| MH_03_medium | 200.0 | 5.0 | 200.0 | 5 | 0.0% | +0.0 | -0.0 | 0.993 / 0.993 | 0.0 |
| MH_04_difficult | 200.0 | 5.0 | 200.0 | 5 | 0.0% | +0.0 | +0.1 | 0.989 / 0.989 | 0.0 |
| MH_05_difficult | 200.0 | 5.0 | 200.0 | 5 | 0.0% | +0.0 | -0.0 | 0.989 / 0.989 | 0.0 |
| V1_01_easy | 200.0 | 5.0 | 200.0 | 5 | 0.0% | +0.0 | -0.1 | 0.990 / 0.990 | 0.0 |
| V1_02_medium | 200.0 | 5.0 | 200.0 | 5 | 0.0% | -0.0 | -0.0 | 0.998 / 0.998 | 0.0 |
| V1_03_difficult | 200.0 | 5.0 | 200.0 | 5 | 0.0% | +0.0 | +0.0 | 0.994 / 0.994 | 0.0 |
| V2_01_easy | 200.0 | 5.0 | 200.0 | 5 | 0.0% | +0.0 | -0.0 | 0.990 / 0.990 | 0.0 |
| V2_02_medium | 200.0 | 5.0 | 200.0 | 5 | 0.0% | -0.0 | -0.0 | 0.998 / 0.998 | 0.0 |
| V2_03_difficult | 200.0 | 5.0 | 200.0 | 5 | 0.0% | -1.5 | -1.7 | 0.996 / 0.996 | 1.1 |
| room1 | 199.4 | 5.0 | 117.3 | 1058 | 1.8% | -1.1 | +1.8 | 0.985 / 0.985 | 1.2 |
| room2 | 199.4 | 5.0 | 107.4 | 2492 | 9.8% | +1.9 | +2.1 | 0.932 / 0.931 | 2.0 |
| room3 | 199.4 | 5.0 | 107.8 | 1992 | 9.0% | +0.1 | -0.2 | 0.951 / 0.951 | 0.1 |
| room4 | 199.4 | 5.0 | 117.4 | 483 | 1.4% | +1.7 | +2.0 | 0.978 / 0.977 | 1.0 |
| room5 | 199.4 | 5.0 | 119.4 | 167 | 0.3% | -0.1 | +0.8 | 0.991 / 0.991 | 0.1 |
| room6 | 199.4 | 5.0 | 116.5 | 3150 | 2.4% | +1.1 | -1.3 | 0.950 / 0.950 | 0.6 |
What it shows, each statement checked by tests/test_timing_page.py:
- No recording has a long IMU gap. The IMU timestamps are regular in every sequence. The EuRoC IMU is exactly 200 Hz; the TUM VI IMU runs at about 199.4 Hz, not 200.
- The EuRoC ground truth is a 200 Hz stream with no dropouts. The TUM VI motion capture is not: it is nominally about 120 Hz, has dropouts in every room, and in two rooms spends roughly a tenth of the recording in them.
- The estimated offsets are small. Every sequence is within a few milliseconds, and the displacement that causes at the median speed is a few millimetres, far below the 0.8 m noise of the simulated GNSS. A ground-truth-to-IMU offset of this size does not affect the results on the EuRoC and TUM VI pages.
- The estimates are meaningful. The correlation at the best shift is high in every sequence, and nearly the same as at zero shift, so the shift is a small correction.
- The two methods agree. The magnitude check, which needs no frame assumption, lands within a few milliseconds of the vector estimate in every sequence.
- The TUM VI authors' time alignment holds up. They aligned the motion capture to the IMU by matching gyroscope and ground-truth rates. An independent estimate by the same principle finds the residual offsets within a few milliseconds.
Limits of the estimate. The EuRoC Machine Hall ground truth is a filtered estimate built from the inertial data, so its offset from the IMU is small by construction and is not an independent check of the clocks. On an exactly aligned synthetic sequence the vector estimate is within half a millisecond and recovers an injected shift to within the same bound; on a recording the real uncertainty is larger and was not estimated. The method needs the platform to turn.
What the dropouts do. Inside a TUM VI dropout the simulated GNSS fixes and the scoring reference follow a straight line between the two nearest samples. For a gap of a second or more that can differ from the true path by a distance comparable to the GNSS noise. The comparison on the TUM VI page is made on identical data for every setting, so the differences between settings are not a product of the dropouts, and the ordering the page reports also appears in the rooms with the fewest dropouts. The absolute errors in rooms 2 and 3 are less trustworthy than in the others.
4. Calibration assumptions¶
| Quantity | Assumed in the harness | Evidence |
|---|---|---|
| IMU-to-body rotation and lever arm | Identity, zero | EuRoC's imu0/sensor.yaml gives an identity body-to-sensor transform. TUM VI's ground truth is already expressed in the IMU frame. |
| GNSS antenna position | At the IMU origin; no lever arm | Simulated GNSS only. A real receiver needs a lever arm and an attitude-dependent correction. |
| Camera-to-IMU extrinsic | Identity | The surrogate emits relative poses directly in the IMU frame. Both datasets ship camera extrinsics; neither is used. |
| Camera intrinsics, distortion, rolling shutter | Not modelled | There are no images in the harness. |
| IMU scale factor, axis misalignment, g-sensitivity | Not modelled | A hypothesis for part of the gap between datasheet and real error growth (ADR-0014); not tested. |
| IMU noise and bias walk | The sensor file's figures, scaled by a tuning where stated | EuRoC's sensor.yaml; TUM VI's imu_config.yaml, which its authors inflated. |
| Gravity | A constant 9.80665 m/s² along the world z axis | A per-latitude and per-altitude value is not used. The frame check on each run tests the direction and norm. |
| Ground truth | The reference, for the start pose and velocity and for scoring | An estimate. EuRoC's notes say its synchronisation with the sensors is limited. |
5. Measured on real data, and assumed in simulation¶
| Measured on a recording | Assumed in simulation | |
|---|---|---|
| Timestamp regularity and rate | Yes (table above) | A uniform grid |
| Ground-truth to IMU offset | Yes, by rate matching | Zero |
| Ground-truth dropouts | Yes | None |
| Bias random walk | Indirectly, by what the filter needs (EuRoC, TUM VI) | The configured value |
| Camera-IMU offset and extrinsic | No | Zero and identity |
| GNSS latency and lever arm | No (no GNSS in either dataset) | Zero |
| Visual front-end error correlation | No (no front end) | Independent, unless a stress option is set |
6. What a real system would still need¶
- A time-offset state, or a calibrated and checked camera-IMU delay, with the estimate carried as uncertainty.
- A GNSS latency model and a lever arm.
- Measured IMU intrinsics, and a decision on whether to estimate them online.
- A measurement of a real visual front end's error correlation, which decides whether the clone can be the default (ADR-0017).
None of these is attempted here.