Content-Based Visual Landmark Search via Multimodal Hypergraph Learning

Zhu, Lei; Shen, Jialie; Jin, Hai; Zheng, Ran; Xie, Liang

doi:10.1109/tcyb.2014.2383389

Cited by 90 publications

(27 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our method is different from them in terms of multi-query construction. Zhu et al [Zhu et al, 2015a] propose to perform landmark classification with a hierarchical multi-modal exemplar features. There are also research [Kennedy and Naaman, 2008] aiming at developing the feature representations for diverse landmark search.…”

Section: Query Argumentation Techniquementioning

confidence: 99%

Effective Multi-Query Expansions: Collaborative Deep Networks for Robust Landmark Retrieval

Wang

Lin

et al. 2017

IEEE Trans. on Image Process.

194

105

View full text Add to dashboard Cite

Given a query photo issued by a user (q-user), the landmark retrieval is to return a set of photos with their landmarks similar to those of the query, while the existing studies on the landmark retrieval focus on exploiting geometries of landmarks for similarity matches between candidate photos and a query photo. We observe that the same landmarks provided by different users over social media community may convey different geometry information depending on the viewpoints and/or angles, and may subsequently yield very different results. In fact, dealing with the landmarks with low quality shapes caused by the photography of q-users is often nontrivial and has seldom been studied. In this paper we propose a novel framework, namely multi-query expansions, to retrieve semantically robust landmarks by two steps. Firstly, we identify the top-k photos regarding the latent topics of a query landmark to construct multi-query set so as to remedy its possible low quality shape . For this purpose, we significantly extend the techniques of Latent Dirichlet Allocation. Then, motivated by the typical collaborative filtering methods, we propose to learn a collaborative deep networks based semantically, nonlinear and high-level features over the latent factor for landmark photo as the training set, which is formed by matrix factorization over collaborative user-photo matrix regarding the multi-query set. The learned deep network is further applied to generate the features for all the other photos, meanwhile resulting into a compact multiquery set within such space. Then, the final ranking scores are calculated over the high-level feature space between the multi-query set and all other photos, which are ranked to serve as the final ranking list of landmark retrieval. Extensive experiments are conducted on real-world social media data with both landmark photos together with their user information to show the superior performance over the existing methods, especially our recently proposed multi-query based mid-level pattern representation method [Wang et al., 2015a].

show abstract

Section: Query Argumentation Techniquementioning

confidence: 99%

Effective Multi-Query Expansions: Collaborative Deep Networks for Robust Landmark Retrieval

Wang

Lin

et al. 2017

IEEE Trans. on Image Process.

194

105

View full text Add to dashboard Cite

show abstract

“…A hypergraph G(D, U, W) is composed of a vertex set D, a hyperedge set U, and a diagonal matrix of hyperedge weights W [42]. Here, U is a family of hyperedges e that connect arbitrary subsets of D, and each hyperedge e is assigned a weight W(e).…”

Section: A Medical Expertise Distributionmentioning

confidence: 99%

Multiview Physician-Specific Attributes Fusion for Health Seeking

Nie¹,

Zhang

Yan

et al. 2017

IEEE Trans. Cybern.

View full text Add to dashboard Cite

Abstract-Community-based health services have risen as important online resources for resolving users health concerns. Despite the value, the gap between what health seekers with specific health needs and what busy physicians with specific attitudes and expertise can offer is being widened. To bridge this gap, we present a question routing scheme that is able to connect health seekers to the right physicians. In this scheme, we first bridge the expertise matching gap via a probabilistic fusion of the physician-expertise distribution and the expertise-question distribution. The distributions are calculated by hypergraphbased learning and kernel density estimation. We then measure physicians attitudes toward answering general questions from the perspectives of activity, responsibility, reputation, and willingness. At last, we adaptively fuse the expertise modeling and attitude modeling by considering the personal needs of the health seekers. Extensive experiments have been conducted on a real-world dataset to validate our proposed scheme.Index Terms-Adaptive fusion, attitude modeling, communitybased health services (CHSs), expertise modeling, question routing.

show abstract

“…With the learned parameters, weak classifier of in modality can be constructed as (2) At each iteration, we calculate total error of weak classifier by summing its prediction errors on all training images (3) where is weight of training image calculated at iteration . We choose weak classifier with the lowest error rate at th iteration (4) After that, the global image corresponding to is added into the global exemplar set in modality . , where , and for incorrect and correct classification respectively.…”

Section: ) Global Exemplar Selectionmentioning

confidence: 99%

Landmark Classification With Hierarchical Multi-Modal Exemplar Feature

Zhu

Shen

Jin

et al. 2015

IEEE Trans. Multimedia

Self Cite

View full text Add to dashboard Cite

Landmark image classification attracts increasing research attention due to its great importance in real applications, ranging from travel guide recommendation to 3-D modelling and visualization of geolocation. While large amount of efforts have been invested, it still remains unsolved by academia and industry. One of the key reasons is the large intra-class variance rooted from the diverse visual appearance of landmark images. Distinguished from most existing methods based on scalable image search, we approach the problem from a new perspective and model landmark classification as multi-modal categorization, which enjoys advantages of low storage overhead and high classification efficiency. Toward this goal, a novel and effective feature representation, called hierarchical multi-modal exemplar (HMME) feature, is proposed to characterize landmark images. In order to compute HMME, training images are first partitioned into the regions with hierarchical grids to generate candidate images and regions. Then, at the stage of exemplar selection, hierarchical discriminative exemplars in multiple modalities are discovered automatically via iterative boosting and latent region label mining. Finally, HMME is generated via a region-based locality-constrained linear coding (RLLC), which effectively encodes semantics of the discovered exemplars into HMME. Meanwhile, dimension reduction is applied to reduce redundant information by projecting the raw HMME into lower-dimensional space. The final HMME enjoys advantages of discriminative and linearly separable. Experimental study has been carried out on real world landmark datasets, and the results demonstrate the superior performance of the proposed approach over several state-of-the-art techniques.Index Terms-Dimension reduction, diverse visual contents, exemplar selection, hierarchical multi-modal exemplar feature (HMME), landmark classification, region-based locality-constrained linear coding (RLLC).

show abstract

Content-Based Visual Landmark Search via Multimodal Hypergraph Learning

Cited by 90 publications

References 48 publications

Effective Multi-Query Expansions: Collaborative Deep Networks for Robust Landmark Retrieval

Effective Multi-Query Expansions: Collaborative Deep Networks for Robust Landmark Retrieval

Multiview Physician-Specific Attributes Fusion for Health Seeking

Landmark Classification With Hierarchical Multi-Modal Exemplar Feature

Contact Info

Product

Resources

About