On the Two-View Geometry of Unsynchronized Cameras

Abstract

We present new methods for simultaneously estimating camera geometry and time shift from video sequences from multiple unsynchronized cameras. Algorithms for simultaneous computation of a fundamental matrix or a homography with unknown time shift between images are developed. Our methods use minimal correspondence sets (eight for fundamental matrix and four and a half for homography) and therefore are suitable for robust estimation using RANSAC. Furthermore, we present an iterative algorithm that extends the applicability on sequences which are significantly unsynchronized, finding the correct time shift up to several seconds. We evaluated the methods on synthetic and wide range of real world datasets and the results show a broad applicability to the problem of camera synchronization.

[Papers] Multi-view video synchronization using motion rhythms of human joints

“…We assume the existence of local minima can cause the accuracy to deteriorate, although it is known to be difficult to prove through experiments. Among the epipolar-based approaches, Albl et al's method 17) is known as one of the best ones in the case of large temporal offset.…”

“…We propose a motion-rhythm-based approach that focuses on the temporal structure of 2D human poses because the temporal structure is robust against detection errors and achieves consistency in a wide baseline setup. We also propose a hybrid approach consists of the motionrhythm-based approach and Albl et al's method 17) to get precise sub-frame accuracy results.…”

“…Our motion-rhythm-based approach is robust against detection errors with 2D poses and synchronizes the multi-view videos even in a frame precision with a large initial time offset. Since the epipolar-geometry-based approach 17) can synchronize them in sub-frame precision with a small initial time offset, we combine these approaches to realize a hybrid algorithm that synchronizes in sub-frame precision even with noised 2D poses and a large initial time offset.…”

Motion-based pose estimation via free falling

“…For example the 3D checkerboard and the special room space with orthogonal vanishing points can be used to estimate the intrinsic camera parameters(K)camera rotation(R) and translation(T) [1]. What's more, a 3D video sequences from multiple unsynchronized cameras can be used to compute the fundamental matrix or homography matrix [2](or more generally, a parallelepiped structure in the building). Through these methods can achieve a good robustness and precision, it always need a calibration object with known geometry structure.…”

“…Methods using video sequences from multiple unsynchronized cameras to find the homography matrix and fundamental matrix with the time offset were presented [2]. In [8], [9], a method that can compute the relative geometry relationship by the image point trajectories with time-shift, but the proposed can achieve satisfied results only when good initialization parameters are selected before the optimization algorithms [2]. We present a new solution to calibrate multi camera using multi free fall motion.…”