Deep Learning of Convolutional Auto-Encoder for Image Matching and 3D Object Reconstruction in the Infrared Range

Knyaz, V. A.; Выголов, О. В.; Kniaz, V. V.; Vizilter, Yu. V.; Gorbatsevich, V. S.; Lühmann, Thomas; Conen, N.

doi:10.1109/iccvw.2017.252

Cited by 26 publications

(18 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Accurate image patch matching is required in many applications of photogrammetry and computer vision. For example, local patch descriptors must provide high robustness for an accurate 3D model generation using structure-from-motion (SfM) algorithms (Remondino et al, 2014, Knyaz et al, 2017. Nowadays, local patch descriptors became the main approach for robust point matching.…”

Section: Related Workmentioning

confidence: 99%

“…The approach is based on a convolutional auto-encoder (CAE) (Goodfellow et al, 2016). In previous research (Knyaz et al, 2017) performed by authors, the CAE-based approach was successfully modified for matching local image patches in infrared images. The present paper is focused on the modification of the developed method for effective matching of the RGB local patches in low-textured images.…”

Section: Related Workmentioning

confidence: 99%

“…A CAE architecture proposed in the previous research (Knyaz et al, 2017) was used as a starting point. Three contributions were made: (1) a convolutional layer and a deconvolutional layer were added to process a larger input patch, (2) two fully-connected layers were removed, (3) the number of the input's channels was increased to three.…”

Section: Convolutional Auto-encodersmentioning

confidence: 99%

“…We perform local patch matching using a modified majority vote approach (Knyaz et al, 2017). Let Iq be a local image patch on the query image Q.…”

Section: Local Patch Matchingmentioning

confidence: 99%

“…The architecture is designed to work with the color images. It is based on the previous work for image matching in the infrared range (Knyaz et al, 2017). The auto-encoder takes a color image patch of 64×64 pixels as an input x.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Deep Learning for Lowtextured Image Matching

Kniaz

Федоренко

Фомин

2018

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Commission II, WG II/8 KEY WORDS: image matching, deep convolutional neural networks, auto-encoders, cultural heritage ABSTRACT:Low-textured objects pose challenges for an automatic 3D model reconstruction. Such objects are common in archeological applications of photogrammetry. Most of the common feature point descriptors fail to match local patches in featureless regions of an object. Hence, automatic documentation of the archeological process using Structure from Motion (SfM) methods is challenging. Nevertheless, such documentation is possible with the aid of a human operator. Deep learning-based descriptors have outperformed most of common feature point descriptors recently. This paper is focused on the development of a new Wide Image Zone Adaptive Robust feature Descriptor (WIZARD) based on the deep learning. We use a convolutional auto-encoder to compress discriminative features of a local path into a descriptor code. We build a codebook to perform point matching on multiple images. The matching is performed using the nearest neighbor search and a modified voting algorithm. We present a new "Multi-view Amphora" (Amphora) dataset for evaluation of point matching algorithms. The dataset includes images of an Ancient Greek vase found at Taman Peninsula in Southern Russia. The dataset provides color images, a ground truth 3D model, and a ground truth optical flow. We evaluated the WIZARD descriptor on the "Amphora" dataset to show that it outperforms the SIFT and SURF descriptors on the complex patch pairs.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Convolutional Auto-encodersmentioning

confidence: 99%

“…We perform local patch matching using a modified majority vote approach (Knyaz et al, 2017). Let Iq be a local image patch on the query image Q.…”

Section: Local Patch Matchingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%