Joint vanishing point extraction and tracking

Abstract

We present a novel vanishing point (VP) detection and tracking algorithm for calibrated monocular image sequences. Previous VP detection and tracking methods usually assume known camera poses for all frames or detect and track separately. We advance the state-of-the-art by combining VP extraction on a Gaussian sphere with recent advances in multi-target tracking on probabilistic occupancy fields. The solution is obtained by solving a Linear Program (LP). This enables the joint detection and tracking of multiple VPs over sequences. Unlike existing works we do not need known camera poses, and at the same time avoid detecting and tracking in separate steps. We also propose an extension to enforce VP orthogonality. We augment an existing video dataset consisting of 48 monocular videos with multiple annotated VPs in 14448 frames for evaluation. Although the method is designed for unknown camera poses, it is also helpful in scenarios with known poses, since a multi-frame approach in VP detection helps to regularize in frames with weak VP line support.

The Manhattan Frame Model—Manhattan World Inference in the Space of Surface Normals

“…2D image-space There is a vast literature on VP estimation from RGB images. The goals for VP estimation range from single-image scene parsing [15] and 3D reconstruction [16], [17], [18], [19], VP direction estimation for rotation estimation with respect to man-made environments [1], [20], [21], [22] to VP direction tracking over time to estimate camera rotation and scene structure [23], [24], [25], [26], [27].…”

“…Generally, VPs are extracted by intersecting lines in the image. These intersections are often found after mapping lines to the unit sphere [23], [29], [30], or into other accumulator spaces [31]. Introduced in [15], horizon estimation has emerged as a benchmark for VP estimation algorithms [32], [33].…”

“…The approach by Antunes et al [38] infers the full MMF from RGB images. Relaxing the assumptions about the scene, VPs can be extracted independently [15], [21], [23], [24], [28], [30], [31], [33], [39] akin to the SCW assumption. 3D representations There are many approaches that rely purely on 3D representations of surfaces and scenes.…”

“…Working in the log-domain, the MAP estimate for R is R = arg min R − log p(R|q, z; Σ) := arg min R f (R) . (23) With the posterior in Eq. (22), the cost function f (R) is…”

A Preliminary Work on Layout Slam for Reconstruction of Indoor Corridor Environments

“…The plane Q is spanned by the center of projection at [0 0 0] T . The vanishing direction (D) of line segments l 1 , l 2 can be estimated from the intersection of their interpretation planes Q 1 , Q 2 on the Gaussian sphere (Kroeger et al, 2015). Vanishing direction on 3D space d ∈ R 3 can be defined through homogeneous coordinates D ∈ P 3 .…”