Integrated Content and Context Analysis for Mobile Landmark Recognition

Chen, Tao; Yap, Kim–Hui; Chau, Lap-Pui

doi:10.1109/tcsvt.2011.2161413

Cited by 24 publications

(11 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Chen et al [2011a] addressed the problem of city-scale landmark recognition from cell phone images. More advanced content and context integration techniques for mobile landmark recognition have been proposed to achieve better performance as well [Chen et al 2011b;Li and Yap 2012].…”

Section: Related Workmentioning

confidence: 99%

Content vs. Context

Yin

Seo

Zimmermann

2015

ACM Trans. Multimedia Comput. Commun. Appl.

View full text Add to dashboard Cite

Due to the ubiquity of sensor-equipped smartphones, it has become increasingly feasible for users to capture videos together with associated geographic metadata, for example the location and the orientation of the camera. Such contextual information creates new opportunities for the organization and retrieval of georeferenced videos. In this study we explore the task of landmark retrieval through the analysis of two types of state-of-the-art techniques, namely media-content-based and geocontext-based retrievals. For the contentbased method, we choose the Spatial Pyramid Matching (SPM) approach combined with two advanced coding methods: Sparse Coding (SC) and Locality-Constrained Linear Coding (LLC). For the geo-based method, we present the Geo Landmark Visibility Determination (GeoLVD) approach which computes the visibility of a landmark based on intersections of a camera's field-of-view (FOV) and the landmark's geometric information available from Geographic Information Systems (GIS) and services. We first compare the retrieval results of the two methods, and discuss the strengths and weaknesses of each approach in terms of precision, recall and execution time. Next we analyze the factors that affect the effectiveness for the content-based and the geo-based methods, respectively. Finally we propose a hybrid retrieval method based on the integration of the visual (content) and geographic (context) information, which is shown to achieve significant improvements in our experiments. We believe that the results and observations in this work will enlighten the design of future geo-referenced video retrieval systems, improve our understanding of selecting the most appropriate visual features for indexing and searching, and help in selecting between the most suitable methods for retrieval based on different conditions. ACM Reference Format:Yifang Yin, Beomjoo Seo, and Roger Zimmermann. 2015. Content vs. context: Visual and geographic information use in video landmark retrieval. ACM Trans. Multimedia Comput.

show abstract

Section: Related Workmentioning

confidence: 99%

Content vs. Context

Yin

Seo

Zimmermann

2015

ACM Trans. Multimedia Comput. Commun. Appl.

View full text Add to dashboard Cite

show abstract

“…Therefore, in our experimental evaluation, we have created a landmark database consisting of 3622 training images and 534 testing images using 50 categories of landmarks from the campus in Nanyang Technological University (NTU) [34]. During the construction of the database, the definition of a landmark is taken as a building, structure, or place-of-interest that is unique or distinctive.…”

Section: A Experimental Setupmentioning

confidence: 99%

“…These works can be categorized into two classes of image representation: 1) local patch image representation [3]- [5], [44] that uses visual features extracted from the local patches in the image for recognition; and 2) bag-of-words (BoW) histogram representation [7]- [12], [34] that generates a BoW histogram for each image through vector quantization. In [6], both patches and BoW (texton histogram) for recognition are used.…”

Section: Introductionmentioning

confidence: 99%

Discriminative BoW Framework for Mobile Landmark Recognition

Chen

Yap

2014

IEEE Trans. Cybern.

View full text Add to dashboard Cite

This paper proposes a new soft bag-of-words (BoW) method for mobile landmark recognition based on discriminative learning of image patches. Conventional BoW methods often consider the patches/regions in the images as equally important for learning. Amongst the few existing works that consider the discriminative information of the patches, they mainly focus on selecting the representative patches for training, and discard the others. This binary hard selection approach results in underutilization of the information available, as some discarded patches may still contain useful discriminative information. Further, not all the selected patches will contribute equally to the learning process. In view of this, this paper presents a new discriminative soft BoW approach for mobile landmark recognition. The main contribution of the method is that the representative and discriminative information of the landmark is learned at three levels: patches, images, and codewords. The patch discriminative information for each landmark is first learned and incorporated through vector quantization to generate soft BoW histograms. Coupled with the learned representative information of the images and codewords, these histograms are used to train an ensemble of classifiers using fuzzy support vector machine. Experimental results on two different datasets show that the proposed method is effective in mobile landmark recognition.

show abstract

“…This comparison task can be directly achieved by background subtraction [1], [40], motion comparison [20], or feature matching [21], [38], [39], [41]. For example, Pilet, Strecha, and Fua [40] modeled the background as a Gaussian mixture model whose parameters were estimated by the EM algorithm to compare scenes even under sudden illumination changes.…”

Section: Introductionmentioning

confidence: 99%

“…Ebrahimi and Mayol-Cuevas [38] used a dense set of corner correspondences to describe scenes and then proposed an adaptive sampling technique to compare their contents. In [39], Chen et al proposed a landmark-based scheme to build different code books and then performed scene content comparisons from a mobile camera. Dragusu, Mihalache, and Solea [46] used edge contours to match objects on simple background from a robotic arm.…”

Section: Introductionmentioning

confidence: 99%

Abnormal Scene Change Detection from a Moving Camera Using Bags of Patches and Spider-Web Map

et al. 2014

View full text Add to dashboard Cite

This paper proposes a novel surveillance system for detecting exceptional scene changes as abnormal events with a mobile camera mounted on a robot. In contrast to abnormal event analysis using fixed cameras, three key problems should be tackled in this system, i.e., scene construction, robot localization, and scene comparison.For the first problem, "scene construction", a clustering scheme is proposed for extracting a set of key frames from the surveillance environment. Each key frame is further divided into a set of patches, which forms a sparse representation for representing scene contents. In addition to the compression effect, the scheme can tackle the effects of misalignment and lighting changes well. For the localization problem, a novel patch matching method is proposed to reduce not only the size of the search space but also the size of the feature dimensions in similarity matching. To prune the search space, a set of projection kernels is used to construct a ring structure. Then, one order of time complexity in the similarity calculation can be reduced from the structure. After scene searching, the robot location is not always guaranteed to be successfully registered to the scene map. Thus, a novel spider-web map is proposed to tackle the effect of misalignment and then detect different exceptional scene changes from the videos. The proposed method has been rigorously tested on a variety of videos to demonstrate its superiority in object detection and abnormal scene change detection.Index Terms-behavior analysis, abnormal scene change detection, pattern matching, video surveillance 1530-437X (c)

show abstract

Integrated Content and Context Analysis for Mobile Landmark Recognition

Cited by 24 publications

References 28 publications

Content vs. Context

Content vs. Context

Discriminative BoW Framework for Mobile Landmark Recognition

Abnormal Scene Change Detection from a Moving Camera Using Bags of Patches and Spider-Web Map

Contact Info

Product

Resources

About