Landmark Classification With Hierarchical Multi-Modal Exemplar Feature

Abstract

Landmark image classification attracts increasing research attention due to its great importance in real applications, ranging from travel guide recommendation to 3-D modelling and visualization of geolocation. While large amount of efforts have been invested, it still remains unsolved by academia and industry. One of the key reasons is the large intra-class variance rooted from the diverse visual appearance of landmark images. Distinguished from most existing methods based on scalable image search, we approach the problem from a new perspective and model landmark classification as multi-modal categorization, which enjoys advantages of low storage overhead and high classification efficiency. Toward this goal, a novel and effective feature representation, called hierarchical multi-modal exemplar (HMME) feature, is proposed to characterize landmark images. In order to compute HMME, training images are first partitioned into the regions with hierarchical grids to generate candidate images and regions. Then, at the stage of exemplar selection, hierarchical discriminative exemplars in multiple modalities are discovered automatically via iterative boosting and latent region label mining. Finally, HMME is generated via a region-based locality-constrained linear coding (RLLC), which effectively encodes semantics of the discovered exemplars into HMME. Meanwhile, dimension reduction is applied to reduce redundant information by projecting the raw HMME into lower-dimensional space. The final HMME enjoys advantages of discriminative and linearly separable. Experimental study has been carried out on real world landmark datasets, and the results demonstrate the superior performance of the proposed approach over several state-of-the-art techniques.Index Terms-Dimension reduction, diverse visual contents, exemplar selection, hierarchical multi-modal exemplar feature (HMME), landmark classification, region-based locality-constrained linear coding (RLLC).

GLNMDA: a novel method for miRNA-disease association prediction based on global linear neighborhoods

“…It is worth noting that to guarantee the convergence of the iterative update rule, we need to normalize our training data in advance[67,68]. Besides, to get an informative value of k for matrix factorization, we employed the clusterONE algorithm accordingly[69], a method for detecting potentially overlapping protein complexes from protein-protein interaction networks.…”

Part-Aware Fine-Grained Object Categorization Using Weakly Supervised Part Detection Network

“…F INE-grained object categorization aims for distinguishing objects of subordinate categories that belong to the same entry-level object category, e.g., various species of birds [1], [2], [3], dogs [4], or flowers [5]. Owing to its importance in multimedia information retrieval [6], [7], [8], fine-grained object categorization has attracted widespread attention from multimedia community. However, the fine-grained categorization tasks are challenging because the variations among different subordinate object categories are subtle, which are often overwhelmed by those caused by arbitrary poses, viewpoint change, and/or occlusion.…”

Augmented Collaborative Filtering for Sparseness Reduction in Personalized POI Recommendation

“…1 The increasing availability of geotagged photos opens up new opportunities in a wide variety of location-oriented multimedia research and applications, including georeferenced image search [Zhu et al 2015a[Zhu et al , 2016Cheng and Shen 2016], location estimation [Li et al 2015b], as well as scene visualization [Rudinac et al 2013].…”