Measuring Digital System Latency from Sensing to Actuation at Continuous 1-ms Resolution

Abstract

This paper describes a new method for measuring the end-to-end latency between sensing and actuation in a digital computing system. Compared to previous works, which generally measured the latency at 10-33-ms intervals or at discrete events separated by hundreds of ms, our new method measures the latency continuously at 1-ms resolution. This allows for the observation of variations in latency over sub 1-s periods, instead of relying upon averages of measurements. We have applied our method to two systems, the first using a camera for sensing and an LCD monitor for actuation, and the second using an orientation sensor for sensing and a motor for actuation. Our results show two interesting findings. First, a cyclical variation in latency can be seen based upon the relative rates of the sensor and actuator clocks and buffer times; for the components we tested, the variation was in the range of 15-50 Hz with a magnitude of 10-20 ms. Second, orientation sensor error can look like a variation in latency; for the sensor we tested, the variation was in the range of 0.5-1.0 Hz with a magnitude of 20-100 ms. Both of these findings have implications for robotics and virtual reality systems. In particular, it is possible that the variation in apparent latency caused by orientation sensor error may have some relation to simulator sickness.

The effects of 0.2Hz varying latency with 20–100ms varying amplitude on simulator sickness in a helmet mounted display

“…A study by Wu et al [8] measured the latency of a system using a magnetometer-accelerometer-gyroscope sensor of the type for tracking orientation that is commonly used in HMDs. Unlike previous works which relied upon averages of measurements, the latency was measured continuously at 1 ms resolution in order to observe how it varied over time.…”

Measuring Latency in Virtual Environments

“…All camera-based systems rely on the latency being higher than the Nyquist rate of the camera, or no latency will be detected at all. One way to negate these issues is to simply use a high speed camera, though then the cost advantages diminish and due to the short exposure time, illumination becomes a concern [30]. Alternatively, there are techniques that derive latency independently of the frame.…”

“…For the angular difference method, they implemented an algorithm which assessed the angle of two bars and calculated the difference between them. Their measurements were accurate enough to reconstruct the apparent oscillation of the latency over time, due to the interplay between the different clocks of the actuator and the sensor [30].…”

“…Given the small exposure times required for such high capture rates, illumination becomes a concern [30,26]. Lighting fluctuations or slight instabilities in the camera mount or the tracked objects are manifested as small oscillations (noise) in the apparent position of the object in the video.…”

“…This requires a perfect, known linear relationship between the tracker and target position however. Wu et al, managed this by using a video feed of the tracker as the actuator [30]. He et al, measure a real system using a similar technique but with a grid for guidance.…”