Robust and Globally Optimal Manhattan Frame Estimation in Near Real Time

Abstract

Most man-made environments, such as urban and indoor scenes, consist of a set of parallel and orthogonal planar structures. These structures are approximated by the Manhattan world assumption, of which notion can be represented as a Manhattan Frame (MF). Given a set of inputs such as surface normals or vanishing points, we pose an MF estimation problem as a consensus set maximization that maximizes the number of inliers over the rotation search space. Conventionally this problem can be solved by a branch-and-bound framework which mathematically guarantees global optimality. However, the computational time of the conventional branch-and-bound algorithms is rather far from real-time. In this paper, we propose a novel bound computation method on an efficient measurement domain for MF estimation, i.e., the extended Gaussian image (EGI). By relaxing the original problem, we can compute the bound with a constant complexity, while preserving global optimality. Furthermore, we quantitatively and qualitatively demonstrate the performance of the proposed method for various synthetic and real-world data. We also show the versatility of our approach through three different applications: extension to multiple MF estimation, 3D rotation based video stabilization and vanishing point estimation (line clustering).

A fast manhattan frame estimation method based on normal vectors

“…In the experiments, the state‐of‐the‐art MF estimation methods, EGI‐BnB (Joo et al, 2018), MMF (Straub et al, 2014) and RTMF (Straub et al, 2015) are selected as baseline. For a fair comparison, we use the open source code provided by the original authors and use the default values of the associated parameters.…”

“…Referring to (Joo et al, 2018), seven different values of are taken, making the values of be 0.0012, 0.0025, 0.005, 0.01, 0.02, 0.04 and 0.08 respectively. Taking the number of the inliers as 300000, and the number of the outliers as 20000, then 700 normal vector datasets are generated.…”

“…In recent years, many methods (Furukawa et al, 2009; Ghanem et al, 2015; Gupta et al, 2013; Taylor & Cowley, 2013; Zhang et al, 2010) have been proposed for MF estimation, but these methods only provide suboptimal solutions (Joo et al, 2018). To ensure the global optimal result, Bazin et al (2012a) and Parra Bustos et al (2014) proposed new methods based on Branch‐and‐Bound (Bazin et al, 2012b) (BnB), but the amount of the calculation of these methods is very large.…”

“…To ensure the global optimal result, Bazin et al (2012a) and Parra Bustos et al (2014) proposed new methods based on Branch‐and‐Bound (Bazin et al, 2012b) (BnB), but the amount of the calculation of these methods is very large. To reduce the amount of computations, Joo et al (2018) proposed a BnB method based on Extended Gaussian Image (EGI). This method has reliable real‐time performance, but it needs to adjust the resolution of EGI to achieve the balance of accuracy and computing speed.…”

Fast regularity-constrained plane fitting

“…Although the method achieved good performance with cluttered scenes, it was time-consuming because of the need to solve large-scale mixed-integer programming problems. More recently, Joo et al [17] presented a fast branch-and-bound framework to estimate the optimal Manhattan frame.…”