Precise timestamping and temporal synchronization in multi-sensor fusion

Abstract

This paper presents a new approach to exact timestamping of asynchronous measurements in a multi-sensor setup. In order to improve the performance of a single sensor, driver assistance systems use several different sensors which have different latencies that usually cannot be measured directly. These unknown latencies pose a problem in data association and temporal synchronization. Consequently, a method to estimate or incorporate the latencies is needed in all sensor fusion algorithms in order to derive the real time of a measurement. In this paper a method is described to compensate the sensor latencies even if they cannot be observed directly.

Temporal synchronization in multi-sensor fusion for future driver assistance systems

“…The estimation of the cycle duration and the minimization of the sensor latencies are presented in detail in [7]. These techniques are restated briefly for the sake of completeness.…”

“…A graphical presentation can be found in Figure 1, where u 1 denotes the unknown inner sensor clock and u 2 the clock of the data acquisition system. For more details and evaluations of this minimization procedure see [7].…”

“…In [6] a parallel hardware device is used to timestamp the measurements directly with GPS accuracy, but this method cannot be applied to all sensors. In [7] a method to filter the timestamps and minimize the sensor latencies is proposed. Yet the possibility of framedrops is not considered, thus a handling of framedrops is needed in addition.…”

Describing a view alignment framework in 3D Tele-immersion systems

“…tp [1] ts [1] ts [2] ts [3] ts [4] ts [5] ts [6] ts [7] ts [8] ts [9] Current Local Time : tc Accurate Window Responsive Window Intersection Window tp [3] Responsive Window ts [4] ts [5] ts [6] ts [7] ts [8] ts [9] ts Figure 5. Aligning the frames adding increasing values of response delays to the rendering, and gauging the user's response on whether the experience was normal, tolerable, slightly tolerable, slightly intolerable and not tolerable.…”

“…These works focus on minimizing skew and maintaining the synchronization between video and audio data across the sender-receiver framework under varying network dynamics. [4] and [5] study synchronizing acquisition of data between multiple sensors, but they either have a high cost of implementation or are not feasible in indoor systems. Our solution considers a very important user experience factor in the buffering and view alignment strategy, which the current methods only partially address.…”