Efficient region of visual interests search for geo-multimedia data

Zhang, Chengyuan; Lin, Yunwu; Zhu, Lei; Zhang, Zuping; Tang, Yan; Huang, Fang

doi:10.1007/s11042-018-6750-6

Cited by 8 publications

(10 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cross-modal retrieval is a significant problem in the area of multimedia computing [19][20][21][22][23][24][25][26], which aims to find out the similar enough objects of one modality in the multimedia database by a query of different modality. Due to the exponential growth of amount of multimedia data, this task attracts a large number of attentions in recent years.…”

Section: Cross-modal Retrievalmentioning

confidence: 99%

Adversarial Learning Based Semantic Correlation Representation for Cross-Modal Retrieval

Zhu¹,

Song²,

Wei³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

With the rapid development of Internet and the widely usage of smart devices, massive multimedia data are generated, collected, stored and shared on the Internet. This trend makes cross-modal retrieval problem become a hot issue in this years. Many existing works pay attentions on correlation learning to generate a common subspace for cross-modal correlation measurement, and others uses adversarial learning technique to abate the heterogeneity of multi-modal data. However, very few works combine correlation learning and adversarial learning to bridge the inter-modal semantic gap and diminish cross-modal heterogeneity. This paper propose a novel cross-modal retrieval method, named ALSCOR, which is an end-to-end framework to integrate cross-modal representation learning, correlation learning and adversarial. CCA model, accompanied by two representation model, VisNet and TxtNet is proposed to capture non-linear correlation. Beside, intra-modal classifier and modality classifier are used to learn intra-modal discrimination and minimize the inter-modal heterogeneity. Comprehensive experiments are conducted on three benchmark datasets. The results demonstrate that the proposed ALSCOR has better performance than the state-of-the-arts.

show abstract

Section: Cross-modal Retrievalmentioning

confidence: 99%

Adversarial Learning Based Semantic Correlation Representation for Cross-Modal Retrieval

Zhu¹,

Song²,

Wei³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…It is obvious that the deep learning based methods have much better performance than the traditional hand-crafted feature based methods. In our previous works [45], [46], we proposed to combine the spatial search techniques and visual feature representations to solve geo-multimedia retrieval problem. However, as far as we know there is no existing image retrieval approach that is suitable to address the reverse spatial visual query (RSVQ) problem.…”

Section: A Image Retrievalmentioning

confidence: 99%

“…For the visual representation of geo-image, we utilize the hand-crafted features, namely SIFT descriptor, and combining with BoVW model to encode the visual content, which is a conventional way used in many image search tasks [46], [75], [76]. Specifically, the visual features are extracted by SIFT technique and clustered by k-means method to generate visual dictionary.…”

Section: B Baseline Introductionmentioning

confidence: 99%

Reverse Spatial Visual Top-$k$ Query

Zhu

Song

et al. 2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

With the wide application of mobile Internet techniques an location-based services (LBS), massive multimedia data with geo-tags has been generated and collected. In this paper, we investigate a novel type of spatial query problem, named reverse spatial visual top-k query (RSVQ k) that aims to retrieve a set of geo-images that have the query as one of the most relevant geo-images in both geographical proximity and visual similarity. Existing approaches for reverse top-k queries are not suitable to address this problem because they cannot effectively process unstructured data, such as image. To this end, firstly we propose the definition of RSVQ k problem and introduce the similarity measurement. A novel hybrid index, named VR 2-Tree is designed, which is a combination of visual representation of geo-image and R-Tree. Besides, an extension of VR 2-Tree, called CVR 2-Tree is introduced and then we discuss the calculation of lower/upper bound, and then propose the optimization technique via CVR 2-Tree for further pruning. In addition, a search algorithm named RSVQ k algorithm is developed to support the efficient RSVQ k query. Comprehensive experiments are conducted on four geo-image datasets, and the results illustrate that our approach can address the RSVQ k problem effectively and efficiently. INDEX TERMS Geo-image, reverse top-k query, spatial visual query, hybrid index.

show abstract

“…Content-based image retrieval (CBIR for short) [43][44][45][46] is to retrieve images by analyzing visual contents, and therefore image representation [18,47] plays an important role in this task. In recent years, the task of CBIR has attracted more and more attentions in the multimedia [21,48,49] and computer vision community [19,20].…”

Section: Related Workmentioning

confidence: 99%

Efficient Multimedia Similarity Measurement Using Similar Elements

Long¹,

Zhu²,

Yuan³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Online social networking techniques and large-scale multimedia retrieval are developing rapidly, which not only has brought great convenience to our daily life, but generated, collected, and stored large-scale multimedia data as well. This trend has put forward higher requirements and greater challenges on massive multimedia retrieval. In this paper, we investigate the problem of image similarity measurement, which is one of the key problems of multimedia retrieval. Firstly, the definition of similarity measurement of images and the related notions are proposed. Then, an efficient similarity measurement framework is proposed. Besides, we present a novel basic method of similarity measurement named SMIN. To improve the performance of similarity measurement, we carefully design a novel indexing structure called SMI Temp Index (SMII for short). Moreover, we establish an index of potential similar visual words off-line to solve to problem that the index cannot be reused. Experimental evaluations on two real image datasets demonstrate that the proposed approach outperforms state-of-the-arts.

show abstract

Efficient region of visual interests search for geo-multimedia data

Cited by 8 publications

References 49 publications

Adversarial Learning Based Semantic Correlation Representation for Cross-Modal Retrieval

Adversarial Learning Based Semantic Correlation Representation for Cross-Modal Retrieval

Reverse Spatial Visual Top-$k$ Query

Efficient Multimedia Similarity Measurement Using Similar Elements

Contact Info

Product

Resources

About