Size Does Matter: Improving Object Recognition and 3D Reconstruction with Cross-Media Analysis of Image Clusters

Gammeter, Stephan; Quack, Till; Tingdahl, David; Gool, Luc Van

doi:10.1007/978-3-642-15549-9_53

Cited by 9 publications

(18 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…5a. As reported by [13], the cluster sizes are power law distributed. We can observe that the distribution shifts towards smaller clusters when more seeds are used.…”

Section: Distribution Of Cluster Sizes and Performance Gapmentioning

confidence: 56%

“…Best Match returns the object with the highest scoring representative, as done in, e.g., [4,6,13,2]. A difference in our case is that we are using a soft clustering, so a representative can belong to multiple objects.…”

Section: Methodsmentioning

confidence: 99%

“…Li et al [16] summarize the input image collection in a set of iconic images by applying k-means clustering based on Gist descriptors, and use only these images to represent a landmark in retrieval. Gammeter et al [13] identify sets of very similar images using complete-link hierarchical agglomerative clustering and replace them by just one image. This step yields a slight compression of the index without loss in performance.…”

Section: Eliminating Redundancymentioning

confidence: 99%

“…Because it has become the de-facto standard in landmark recognition [1,2,13,3], we use the vector space model for image retrieval [15,8]. We extract SIFT [44] features from each image in the dataset and jointly cluster them using approximate k-means [8] to generate a dictionary of 1M visual words.…”

Section: Image Retrievalmentioning

confidence: 99%

See 3 more Smart Citations

Visual landmark recognition from Internet photo collections: A large-scale evaluation

Weyand

Leibe

2015

Computer Vision and Image Understanding

View full text Add to dashboard Cite

The task of a visual landmark recognition system is to identify photographed buildings or objects in query photos and to provide the user with relevant information on them. With their increasing coverage of the world's landmark buildings and objects, Internet photo collections are now being used as a source for building such systems in a fully automatic fashion. This process typically consists of three steps: clustering large amounts of images by the objects they depict; determining object names from user-provided tags; and building a robust, compact, and efficient recognition index. To this date, however, there is little empirical information on how well current approaches for those steps perform in a large-scale open-set mining and recognition task. Furthermore, there is little empirical information on how recognition performance varies for different types of landmark objects and where there is still potential for improvement. With this paper, we intend to fill these gaps. Using a dataset of 500k images from Paris, we analyze each component of the landmark recognition pipeline in order to answer the following questions: How many and what kinds of objects can be discovered automatically? How can we best use the resulting image clusters to recognize the object in a query? How can the object be efficiently represented in memory for recognition? How reliably can semantic information be extracted? And finally: What are the limiting factors in the resulting pipeline from query to semantics? We evaluate how different choices of methods and parameters for the individual pipeline steps affect overall system performance and examine their effects for different query categories such as buildings, paintings or sculptures.

show abstract

“…5a. As reported by [13], the cluster sizes are power law distributed. We can observe that the distribution shifts towards smaller clusters when more seeds are used.…”

Section: Distribution Of Cluster Sizes and Performance Gapmentioning

confidence: 56%

Section: Methodsmentioning

confidence: 99%

Section: Eliminating Redundancymentioning

confidence: 99%

Section: Image Retrievalmentioning

confidence: 99%

See 2 more Smart Citations

Visual landmark recognition from Internet photo collections: A large-scale evaluation

Weyand

Leibe

2015

Computer Vision and Image Understanding

View full text Add to dashboard Cite

show abstract

“…Firstly, using HOP, we can precisely localize even very small details on a large facade since the images in the clusters can help bridge larger scale changes than normal local feature matching can handle. Secondly, as in [8], the clusters provide additional images for each detail showing it under different lighting conditions and viewing angles and can thus be used as an "offline query expansion" to make recognition of these details more robust compared to matching against the Wikipedia images alone.…”

Section: Datasetmentioning

confidence: 99%

Discovering Details and Scene Structure with Hierarchical Iconoid Shift

Weyand

Leibe

2013

2013 IEEE International Conference on Computer Vision

View full text Add to dashboard Cite

Current landmark recognition engines are typically aimed at recognizing building-scale landmarks, but miss interesting details like portals, statues or windows. This is because they use a flat clustering that summarizes all photos of a building facade in one cluster. We propose Hierarchical Iconoid Shift, a novel landmark clustering algorithm capable of discovering such details. Instead of just a collection of clusters, the output of HIS is a set of dendrograms describing the detail hierarchy of a landmark. HIS is based on the novel Hierarchical Medoid Shift clustering algorithm that performs a continuous mode search over the complete scale space. HMS is completely parameter-free, has the same complexity as Medoid Shift and is easy to parallelize. We evaluate HIS on 800k images of 34 landmarks and show that it can extract an often surprising amount of detail and structure that can be applied, e.g., to provide a mobile user with more detailed information on a landmark or even to extend the landmark's Wikipedia article.

show abstract

Learning Where to Classify in Multi-view Semantic Segmentation

Riemenschneider

Bodis

Weissenberg

et al. 2014

Computer Vision – ECCV 2014

Self Cite

View full text Add to dashboard Cite

There is an increasing interest in semantically annotated 3D models, e.g. of cities. The typical approaches start with the semantic labelling of all the images used for the 3D model. Such labelling tends to be very time consuming though. The inherent redundancy among the overlapping images calls for more efficient solutions. This paper proposes an alternative approach that exploits the geometry of a 3D mesh model obtained from multi-view reconstruction. Instead of clustering similar views, we predict the best view before the actual labelling. For this we find the single image part that bests supports the correct semantic labelling of each face of the underlying 3D mesh. Moreover, our singleimage approach may surprise because it tends to increase the accuracy of the model labelling when compared to approaches that fuse the labels from multiple images. As a matter of fact, we even go a step further, and only explicitly label a subset of faces (e.g. 10%), to subsequently fill in the labels of the remaining faces. This leads to a further reduction of computation time, again combined with a gain in accuracy. Compared to a process that starts from the semantic labelling of the images, our method to semantically label 3D models yields accelerations of about 2 orders of magnitude. We tested our multi-view semantic labelling on a variety of street scenes.

show abstract

Size Does Matter: Improving Object Recognition and 3D Reconstruction with Cross-Media Analysis of Image Clusters

Cited by 9 publications

References 25 publications

Visual landmark recognition from Internet photo collections: A large-scale evaluation

Visual landmark recognition from Internet photo collections: A large-scale evaluation

Discovering Details and Scene Structure with Hierarchical Iconoid Shift

Learning Where to Classify in Multi-view Semantic Segmentation

Contact Info

Product

Resources

About