Unfolding an Indoor Origami World

Abstract

Abstract. In this work, we present a method for single-view reasoning about 3D surfaces and their relationships. We propose the use of midlevel constraints for 3D scene understanding in the form of convex and concave edges and introduce a generic framework capable of incorporating these and other constraints. Our method takes a variety of cues and uses them to infer a consistent interpretation of the scene. We demonstrate improvements over the state-of-the art and produce interpretations of the scene that link large planar surfaces.

Patchwork Stereo: Scalable, Structure-Aware 3D Reconstruction in Man-Made Environments

“…Fouhey et al [8] use a scene representation relying on multiple top-down partitions of an image. They intersect sets of 2D rays cast from pairs of vanishing points, defining projective rectilinear superpixels/patches whose boundaries reflects their 3D orientation.…”

Stereo reconstruction using top-down cues

“…Examples of these kinds of relationship at the local level (e.g. dealing with small collections of primitives), include data driven techniques to recognise common configurations of oriented planes [38], or concave/convex edges [5], and recent deep learning approaches which exploit prelearned representations [4]. These data-driven techniques are designed to exploit the recent prevalence of large scale reconstruction datasets, to learn which configurations are the most common and recognisable.…”

“…Finally, an energy (which is linear in the plane parameters) can be formulated by aggregating the inverse depth errors over all the triangulated correspondences, while accounting for their triangulation confidences An example of an indoor scene interpretation in the "Origami world" [5,38]. Left: input image; right: 3D scene interpretation with colour-coded normal directions (at every point the R-G-B colour channels represent the z-x-y components of the vector normal to the surface.)…”

“…We use a data-driven approach (following Fouhey et al . [5]) to probabilistically estimate surface normals and edge classes to create an "Origami world", as shown in Figure 2. This data-driven approach is able to exploit the recent availability of large-scale datasets to learn what scene structures are realistic.…”

“…Studies have found that these types of top-down cues take around 7 months of continuous online training to emerge in humans [3]. It is only recently, with the growth of large-scale 3D datasets that more accurate data-driven approaches to scene understanding have become feasible within computer vision [4,5]. In previous work, some of the most commonly exploited cues involve assumptions about gravity and the viewing orientation, assumptions about the types of surfaces found in urban environments (for example the Manhattan world assumption) and assumptions about common configurations of objects and primitives.…”