From single image query to detailed 3D reconstruction

Schönberger, Johannes L.; Radenović, Filip; Chum, Ondřej; Frahm, Jan‐Michael

doi:10.1109/cvpr.2015.7299148

Cited by 101 publications

(62 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These techniques generally operate on 10s to 1000s of images. Using such methods, past work has used retrieval and SfM to build a 3D model seeded from a single image [31], or registered a photo to an existing 3D model to transfer depth [40]. However, this work requires either having a detailed 3D model of each location in advance, or building one at run-time.…”

Section: Related Workmentioning

confidence: 99%

MegaDepth: Learning Single-View Depth Prediction from Internet Photos

Snavely

2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

776

677

View full text Add to dashboard Cite

Single-view depth prediction is a fundamental problem in computer vision. Recently, deep learning methods have led to significant progress, but such methods are limited by the available training data. Current datasets based on 3D sensors have key limitations, including indoor-only images (NYU), small numbers of training examples (Make3D), and sparse sampling (KITTI). We propose to use multi-view Internet photo collections, a virtually unlimited data source, to generate training data via modern structure-from-motion and multi-view stereo (MVS) methods, and present a large depth dataset called MegaDepth based on this idea. Data derived from MVS comes with its own challenges, including noise and unreconstructable objects. We address these challenges with new data cleaning methods, as well as automatically augmenting our data with ordinal depth relations generated using semantic segmentation. We validate the use of large amounts of Internet data by showing that models trained on MegaDepth exhibit strong generalization-not only to novel scenes, but also to other diverse datasets including Make3D, KITTI, and DIW, even when no images from those datasets are seen during training. 1

show abstract

Section: Related Workmentioning

confidence: 99%

MegaDepth: Learning Single-View Depth Prediction from Internet Photos

Snavely

2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

776

677

View full text Add to dashboard Cite

show abstract

“…We achieve this by exploiting the geometry and the camera positions from 3D models reconstructed automatically by a structure-from-motion (SfM) pipeline. The stateof-the-art retrieval-SfM pipeline [17] takes an unordered image collection as input and attempts to build all possible 3D models. To make the process efficient, fast image clustering is employed.…”

Section: Introductionmentioning

confidence: 99%

Fine-Tuning CNN Image Retrieval with No Human Annotation

Radenović

Tolias

Chum

2019

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

955

1,090

View full text Add to dashboard Cite

Image descriptors based on activations of Convolutional Neural Networks (CNNs) have become dominant in image retrieval due to their discriminative power, compactness of representation, and search efficiency. Training of CNNs, either from scratch or fine-tuning, requires a large amount of annotated data, where a high quality of annotation is often crucial. In this work, we propose to fine-tune CNNs for image retrieval on a large collection of unordered images in a fully automated manner. Reconstructed 3D models obtained by the state-of-the-art retrieval and structure-from-motion methods guide the selection of the training data. We show that both hard-positive and hard-negative examples, selected by exploiting the geometry and the camera positions available from the 3D models, enhance the performance of particular-object retrieval. CNN descriptor whitening discriminatively learned from the same training data outperforms commonly used PCA whitening. We propose a novel trainable Generalized-Mean (GeM) pooling layer that generalizes max and average pooling and show that it boosts retrieval performance. Applying the proposed method to the VGG network achieves state-of-the-art performance on the standard benchmarks: Oxford Buildings, Paris, and Holidays datasets.

show abstract

“…Incremental SfM pipelines [30,35,26] build a 3D model by initializing the structure from a small seed and gradually growing it by adding additional cameras. This scheme is closer to the requested progressive scenario but, unfortunately, is strongly dependent on the order in which images are added to the model [20,27]. View selection algorithms [33] carefully determine the image order usually by employing the global matching information which is not available in the progressive case.…”

Section: Related Workmentioning

confidence: 99%

Progressive Structure from Motion

Locher

Havlena

Gool

2018

Computer Vision – ECCV 2018

View full text Add to dashboard Cite

Structure from Motion or the sparse 3D reconstruction out of individual photos is a long studied topic in computer vision. Yet none of the existing reconstruction pipelines fully addresses a progressive scenario where images are only getting available during the reconstruction process and intermediate results are delivered to the user. Incremental pipelines are capable of growing a 3D model but often get stuck in local minima due to wrong (binding) decisions taken based on incomplete information. Global pipelines on the other hand need the access to the complete viewgraph and are not capable of delivering intermediate results. In this paper we propose a new reconstruction pipeline working in a progressive manner rather than in a batch processing scheme. The pipeline is able to recover from failed reconstructions in early stages, avoids to take binding decisions, delivers a progressive output and yet maintains the capabilities of existing pipelines. We demonstrate and evaluate our method on diverse challenging public and dedicated datasets including those with highly symmetric structures and compare to the state of the art.

show abstract

From single image query to detailed 3D reconstruction

Cited by 101 publications

References 21 publications

MegaDepth: Learning Single-View Depth Prediction from Internet Photos

MegaDepth: Learning Single-View Depth Prediction from Internet Photos

Fine-Tuning CNN Image Retrieval with No Human Annotation

Progressive Structure from Motion

Contact Info

Product

Resources

About