Robust 3D Car Shape Estimation from Landmarks in Monocular Image

Abstract

Input Image Landmarks LocalisationModel Fitting Estimated 3D ShapeFigure 1: The framework for 3D shape estimation. Top: A series of prior 3D shape basis [2]. Bottom: The shape estimation procedure for a given input image.Estimation of the 3D shape of a object from monocular image is an under-determined problem, which becomes harder when the observations are severely contaminated. In this paper, we propose a robust model to estimate 3D shape X from 2D landmarks x ∈ R 2×p with unknown camera pose M. The 3D shape of the object is assumed as a linear combination of predefined shape basisTo estimate s and M, we fit the model by minimizing the error between the observations x and the projected model points MX (as shown in Figure 1).Model. To address the outliers in the observed 2D points, which result from the complex background and illumination conditions, we propose a robust 3D shape estimation model. We explicitly model the outliers with an additional sparse error term E ∈ R 2×p . Thus, the robust model is then formulated aswhere t = [t x ,t y ] T · 1 1×p is the translation, and λ , η are the regularization parameters, and µ is the mean shape. The objective function in (1) is non-convex and non-smooth constrained on Stiefel manifold, where the coupling of the unknown shape representation coefficients s and camera pose M makes it more difficult to be solved. Method. We propose an efficient numerical algorithm based on Alternative Direction Method of Multipliers (ADMM) [1] to solve this problem. With an auxiliary variable V ∈ R 2×3 introduced, the augmented Lagrangian is,where Λ is the multiplier and τ is penalty parameter. We update each block with all the others fixed. Based on some analysis on non-convex optimization of ADMM [3], we set the orthogonality constraints into the smooth sub-problem (V -minimization),The closed-form solution is given by V k+1 = UI 2×3 W T , where U andThe other sub-problems can be easily solved. Both the optimization of M and t admit closed-form solutions. The updating of s is a Lasso-problem, and the sparse error pattern E can be efficiently solved by element-wise soft-thresholding. The convergences of ADMM with more than two blocks cannot be always guaranteed [1], and may be influenced by the update ordering. We set a fixed update ordering that can always lead convergence in our experiments.

ApolloCar3D: A Large 3D Car Instance Understanding Benchmark for Autonomous Driving

“…While only a handful of keypoints can be sufficient solve the EPnP problem, we define 66 semantic keypoints in our dataset, as shown in Fig. 3, which has much higher density than most previous car datasets [57,43]. The redundancy enables more accurate and robust shape-and-pose registration.…”

“…Nevertheless, addressing 3D instance understanding from a single image in an uncontrolled environment is ill-posed and challenging, thus attracting growing attention. With the development of deep CNNs, researchers are able to achieve impressive results with supervised [18,69,43,46,57,54,63,70,6,32,49,38,3,66] or weakly supervised strategies [28,48,24]. Existing works consider to represent an object as a parameterized 3D bounding box [18,54,57,49], coarse wire-frame skeletons [14,32,62,69,68], voxels [9], one-hot selection from a small set of exemplar models [3,45,1], and point clouds [17].…”