Recovering the 3d Pose and Shape of Vehicles From Stereo Images

Abstract

ABSTRACT:The precise reconstruction and pose estimation of vehicles plays an important role, e.g. for autonomous driving. We tackle this problem on the basis of street level stereo images obtained from a moving vehicle. Starting from initial vehicle detections, we use a deformable vehicle shape prior learned from CAD vehicle data to fully reconstruct the vehicles in 3D and to recover their 3D pose and shape. To fit a deformable vehicle model to each detection by inferring the optimal parameters for pose and shape, we define an energy function leveraging reconstructed 3D data, image information, the vehicle model and derived scene knowledge. To minimise the energy function, we apply a robust model fitting procedure based on iterative Monte Carlo model particle sampling. We evaluate our approach using the object detection and orientation estimation benchmark of the KITTI dataset (Geiger et al., 2012). Our approach can deal with very coarse pose initialisations and we achieve encouraging results with up to 82 % correct pose estimations. Moreover, we are able to deliver very precise orientation estimation results with an average absolute error smaller than 4• .

Precise Vehicle Reconstruction for Autonomous Driving Applications

“…For collaborative positioning tasks, the estimation of the orientation is the more sensitive factor: for an exemplary vehicle distance of 10 m, an orientation error of 2.8 • leads to around 50 cm error in positioning compared to the 34 cm resulting from the position estimation error. Using the Refine setting we are able to outperform the orientation estimation results of Coenen et al (2018) by 12.4 % and to achieve an equivalent amount of correct orientations as in Zia et al (2013) but with a 1.5 • smaller average error and without depending on good pose initialisations. Though, it has to be noted that in Zia et al (2013) no stereo information was used.…”

“…However, using scene flow for object detection is computationally expensive. Coenen et al (2018) combine 3D stereo information with image cues to derive an ASM representation for vehicle detections. However, their results show only limited benefit from incorporating the image information.…”

“…In this paper we want to reconstruct vehicles from street level stereo images and precisely recover their 3D pose and shape. Similarly to (Coenen et al, 2018) and (Zia et al, 2013) we make use of an ASM as vehicle shape prior. Based on initial 3D vehicle detections, we make the following contributions in this paper: (1) We extend the ASM of (Zia et al, 2013) by defining two groups of keypoints, one describing the geometrical shape and the other additionally representing distinct vehicle part appearances.…”

“…(2) For the reconstruction of vehicles, we define a probabilistic model, incorporating different types of observations by jointly leveraging features derived from our shape prior, reconstructed 3D data, scene knowledge, and image information. The formulation of our probabilistic model is inspired by the energy formulation in (Coenen et al, 2018). However, we modify this energy function to overcome some problems reported by the authors.…”

“…4 is minimized to find the optimal pose and shape parameters for each detected vehicle. As this function is non-convex and the model parameters are continuous, we adapt the iterative Monte Carlo sampling procedure proposed in (Coenen et al, 2018) to approximate the parameter set for which the energy function becomes minimal. To this end we discretise the target parameters by generating model particles for the vehicle ASM.…”