Visual landmark recognition from Internet photo collections: A large-scale evaluation

Abstract

The task of a visual landmark recognition system is to identify photographed buildings or objects in query photos and to provide the user with relevant information on them. With their increasing coverage of the world's landmark buildings and objects, Internet photo collections are now being used as a source for building such systems in a fully automatic fashion. This process typically consists of three steps: clustering large amounts of images by the objects they depict; determining object names from user-provided tags; and building a robust, compact, and efficient recognition index. To this date, however, there is little empirical information on how well current approaches for those steps perform in a large-scale open-set mining and recognition task. Furthermore, there is little empirical information on how recognition performance varies for different types of landmark objects and where there is still potential for improvement. With this paper, we intend to fill these gaps. Using a dataset of 500k images from Paris, we analyze each component of the landmark recognition pipeline in order to answer the following questions: How many and what kinds of objects can be discovered automatically? How can we best use the resulting image clusters to recognize the object in a query? How can the object be efficiently represented in memory for recognition? How reliably can semantic information be extracted? And finally: What are the limiting factors in the resulting pipeline from query to semantics? We evaluate how different choices of methods and parameters for the individual pipeline steps affect overall system performance and examine their effects for different query categories such as buildings, paintings or sculptures.

CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps

“…A related line of research is landmark recognition, where images are clustered by their geolocations and visual similarity to construct a database of popular landmarks. The database serves as the index of an image retrieval system [28,29,30,31,32,33] or the training data of a landmark classifier [34,35,36]. Cross-view geolocation recognition makes additional use of satellite or aerial imagery to determine query locations [37,38,39,40].…”

Siamese network features for image matching

“…Nevertheless, it allows to suggest that Siamese architecture together with contrastive loss objective is a good choice for learning features for image matching and retrieval tasks. Moreover, using additional relevant datasets [26], [27] during training might further enhance the accuracy and performance of the approach.…”

A Survey on Deep Visual Place Recognition

“…Moreover, the point of view is consistently at the street level, without big changes in the vertical orientation. Non robotics datasets typically are not collected as sequences of frames [22], [23], [38], [69], [80], [81], [116]- [118], [122], [125], [190], [215], [242], [244], [246], [251]. In many cases, they are created by collections of online images, with variable viewpoints [122] 2008 Urban City ∼6k Label Holidays [118] 2008 Outdoor World ∼2k Label Eynsham [21] 2009 Urban City ∼70k GPS St. Lucia [240], [241] 2010 Urban City ∼66k GPS European Cities 50k [22] 2010 Urban Continent ∼50k Label Geotagged StreetView [23] 2010 Urban City ∼17k GPS Rome 16k [242] 2010 Urban City ∼16k Pose Dubrovnik 6k [242] 2010 Urban City ∼6.8k Pose San Francisco [243] 2011 Urban City ∼1.06M GPS Alderley [45] 2012 Urban City ∼31k GPS 7 Scenes [244] 2013 Indoor Building ∼43k Pose Nordland [155] 2013 Outdoor Region ∼143k GPS Google StreetView 62k [114] 2014 Urban City ∼62k GPS Freiburg Across Seasons [192], [245] 2014 Urban City ∼43k GPS Cambridge Landmarks [215] 2015 Urban City ∼10.8k Pose Paris500k [246] 2015 Urban City ∼504k Label Pittsburgh [117] 2015 Urban City ∼278k GPS Landmarks-full [80], [125] 2016 Urban World ∼192k Label NCLT [247] 2016 Outdoor + Indoor...…”