Learned Multi-patch Similarity

Abstract

Estimating a depth map from multiple views of a scene is a fundamental task in computer vision. As soon as more than two viewpoints are available, one faces the very basic question how to measure similarity across >2 image patches. Surprisingly, no direct solution exists, instead it is common to fall back to more or less robust averaging of two-view similarities. Encouraged by the success of machine learning, and in particular convolutional neural networks, we propose to learn a matching function which directly maps multiple image patches to a scalar similarity score. Experiments on several multi-view datasets demonstrate that this approach has advantages over methods based on pairwise patch similarity.

Visual recognition in the wild by sampling deep similarity functions

“…Here, we have tested straight-forward, handcrafted averaging and voting schemes. It may however be interesting to also learn the combination, or even to explore an "early combination" where a multi-way similarity [20] is computed from a test example to a set of multiple exemplars.…”

“…After the advent of modern convolutional networks, the same idea was applied to raw images, e.g., [12,13,14,15,16]. Siamese convolutional branches independently transform two (or more) images A and B into high-level representations that are then merged and transformed further into a learned measure F (A, B) of similarity.…”

Superquadrics Revisited: Learning 3D Shape Parsing Beyond Cuboids

“…The simplest representation for 3D reconstruction from one or more images are 2.5D depth maps as they can be inferred using standard 2D convolutional neural networks [14,18,24,43]. Since depth maps are view-based, these methods require additional post-processing algorithms to fuse information from multiple viewpoints in order to capture the entire object geometry.…”

“…This allows us to state the following: if the first primitive exists, the first primitive will be the one closest to point x i of the target point, if the first primitive does not exist and the second does, then the second primitive is closest to point x i and so forth. More formally, this property can be stated A 3D vector r(η, ω) defines a closed surface in space as η (latitude angle) and ω (longitude angle) change in the given intervals (14). The rigid body transformation T m (x) maps a point from the world coordinate system to the local coordinate system of the m th primitive.…”

“…In the last decade, major breakthroughs in shape extraction were due to deep neural networks coupled with the abundance of visual data. Recent works focus on learning 3D reconstruction using 2.5D [14,16,24,43], volumetric [7,11,13,18,30,42], mesh [12,21] and point cloud [10,27] representations. However, none of the above are sufficiently parsimonious or interpretable to allow for higher-level 3D scene understanding as required by intelligent systems.…”