ROECS: A Robust Semi-direct Pipeline Towards Online Extrinsics Correction of the Surround-view System

Abstract

Generally, a surround-view system (SVS), which is an indispensable component of advanced driving assistant systems (ADAS), consists of four to six wide-angle fisheye cameras. As long as both intrinsics and extrinsics of all cameras have been calibrated, a top-down surround-view with the real scale can be synthesized at runtime from fisheye images captured by these cameras. However, when the vehicle is driving on the road, relative poses between cameras in the SVS may change from the initial calibrated states due to bumps or collisions. In case that extrinsics' representations are not adjusted accordingly, on the surround-view, obvious geometric misalignment will appear. Currently, the researches on correcting the extrinsics of the SVS in an online manner are quite sporadic, and a mature and robust pipeline is still lacking. As an attempt to fill this research gap to some extent, in this work, we present a novel extrinsics correction pipeline designed specially for the SVS, namely ROECS (Robust Online Extrinsics Correction of the Surround-view system). Specifically, a "refined bi-camera error" model is firstly designed. Then, by minimizing the overall "bi-camera error" within a sparse and semi-direct framework, the SVS's extrinsics can be iteratively optimized and become accurate eventually. Besides, an innovative three-step pixel selection strategy is also proposed. The superior robustness and the generalization capability of ROECS are validated by both quantitative and qualitative experimental results. To make the results reproducible, the collected data and the source code have been released at https://cslinzhang.github.io/ROECS/.

Method for the automatic measurement of camera-calibration quality in a surround-view system

“…On the other hand, a robust and mature online extrinsic calibration pipeline is still lacking. Recently, some research projects such as [3][4][5][6][7][8][9][10] among others, have proposed different approaches attempting to fill this research gap. Nevertheless, [3] [8] [9] and [10] are the ones that provide the most encouraging results.…”

“…Recently, some research projects such as [3][4][5][6][7][8][9][10] among others, have proposed different approaches attempting to fill this research gap. Nevertheless, [3] [8] [9] and [10] are the ones that provide the most encouraging results. Concerning [3] [8] and [9], all these projects aim to minimize the photometric discrepancy between adjacent cameras to optimize cameras' extrinsic parameters.…”

“…Nevertheless, [3] [8] [9] and [10] are the ones that provide the most encouraging results. Concerning [3] [8] and [9], all these projects aim to minimize the photometric discrepancy between adjacent cameras to optimize cameras' extrinsic parameters. Moreover, all of them are based on the use of natural images to recalibrate the cameras, so any external object such as a calibration pattern is not required.…”

“…Nevertheless, there exists a research gap on how these poses can be optimally online reestimated. Recently, some projects [3][4][5][6][7][8][9][10] have proposed different solutions which aim to address this problem. However, these works disregard the practical aspects of when it is necessary to reestimate the cameras' relative poses.…”

“Seeing” the Invisible: Under Vehicle Reconstruction (UVR) for Surround View Visualization

“…Advanced Driver Assistance System (ADAS) is becoming more and more popular with various features [1] [2]. Displaying surrounding information of a vehicle for assistive parking and lowspeed maneuvering is one of the ADAS functions and of interest to both academia and industrial communities [3][4] [5]. A classical Surround View System consists of three components: (1) four fisheye c ameras m ounted s urrounding t he vehicle f or i mage c apturing; ( 2) a c entral computing unit where raw video frames from all cameras are processed; and (3) a visualization device, such as a built-in display, where the rendered results from specified p erspectives a re s hown t o the driver.…”