Semantic concept-based query expansion and re-ranking for multimedia retrieval

Natsev, Apostol; Haubold, Alexander; Tešić, Jelena; Xie, Lexing; Yan, Rong

doi:10.1145/1291233.1291448

Cited by 161 publications

(112 citation statements)

References 23 publications

Supporting

Mentioning

111

Contrasting

Order By: Relevance

“…WordNet [28] is for example used to translate the query words in visual concepts [32]. Wikipedia is often successfully used to expand a query in image and video retrieval [19,22].…”

Section: Complex Event Detectionmentioning

confidence: 99%

Knowledge based query expansion in complex multimedia event detection

Boer

Schutte

Kraaij

2015

Multimed Tools Appl

View full text Add to dashboard Cite

A common approach in content based video information retrieval is to perform automatic shot annotation with semantic labels using pre-trained classifiers. The visual vocabulary of state-of-the-art automatic annotation systems is limited to a few thousand concepts, which creates a semantic gap between the semantic labels and the natural language query. One of the methods to bridge this semantic gap is to expand the original user query using knowledge bases. Both common knowledge bases such as Wikipedia and expert knowledge bases such as a manually created ontology can be used to bridge the semantic gap. Expert knowledge bases have highest performance, but are only available in closed domains. Only in closed domains all necessary information, including structure and disambiguation, can be made available in a knowledge base. Common knowledge bases are often used in open domain, because it covers a lot of general information. In this research, query expansion using common knowledge bases ConceptNet and Wikipedia is compared to an expert description of the topic applied to content-based information retrieval of complex events. We run experiments on the Test Set of TRECVID MED 2014. Results show that 1) Query Expansion can improve performance compared to using no query expansion in the case that the main noun of the query could not be matched to a concept detector; 2) Query expansion using expert knowledge is not necessarily better than query expansion using common knowledge; 3) ConceptNet performs slightly better than Wikipedia; 4) Late fusion can slightly improve performance. To conclude, query expansion has potential in complex event detection.

show abstract

“…WordNet [28] is for example used to translate the query words in visual concepts [32]. Wikipedia is often successfully used to expand a query in image and video retrieval [19,22].…”

Section: Complex Event Detectionmentioning

confidence: 99%

Knowledge based query expansion in complex multimedia event detection

Boer

Schutte

Kraaij

2015

Multimed Tools Appl

View full text Add to dashboard Cite

show abstract

“…As opposed to that, most of the CBMR approaches index the database using predefined concept detectors [17,18]. In CBMR, query is represented by set of keywords (text) which are mapped to predefined concept detectors by various methods like statistical corpus analysis [19], using Ontology [20,21] etc. However, our method specifically focuses on retrieving relevant images without computing scores for entire database.…”

Section: Related Workmentioning

confidence: 99%

Image Retrieval Using Eigen Queries

Raval

Tonge

Jawahar

2013

Computer Vision – ACCV 2012

View full text Add to dashboard Cite

Abstract. Category based image search, where the goal is to retrieve images of a specific category from a large database, is becoming increasingly popular. In such a setting, the query is often a classifier. However, the complexity of the classifiers (often SVMs) used for this purpose hinders the use of such a solution in practice. Problem becomes paramount when the database is huge and/or the dimensionality of the feature representation is also very large. In this paper, we address this issue by proposing a novel method which decomposes the query classifier into set of known eigen queries. We use their precomputed results (or scores) for computing the ranked list corresponding to novel queries. We also propose an approximate algorithm which accesses only a fraction of the data to perform fast retrieval. Experiments on various datasets show that our method reports high accuracy and efficiency. Apart from retrieval, the proposed method can also be used to discover interesting new concepts from the given dataset.

show abstract

“…In automatic concept-based video retrieval, the fundamental problem is how to map the user's information need into the space of available concepts in the used concept ontology [7]. The basic approach is to select a small number of concept detectors as active and weight them based either on the performance of the detectors or their estimated suitability for the current query.…”

Section: Concept-based Video Retrievalmentioning

confidence: 99%

“…In [7], Natsev et al divide the methods for automatic selection of concepts into three categories: text-based, visual-example-based, and results-based methods. Text-based methods use lexical analysis of the textual query and resources such as WordNet [8] to map query words into concepts.…”

Section: Concept-based Video Retrievalmentioning

confidence: 99%

See 1 more Smart Citation

Improving Automatic Video Retrieval with Semantic Concept Detection

Koskela¹,

Sjöberg²,

Laaksonen³

2009

Image Analysis

View full text Add to dashboard Cite

Abstract. We study the usefulness of intermediate semantic concepts in bridging the semantic gap in automatic video retrieval. The results of a series of large-scale retrieval experiments, which combine text-based search, content-based retrieval, and concept-based retrieval, is presented. The experiments use the common video data and sets of queries from three successive TRECVID evaluations. By including concept detectors, we observe a consistent improvement on the search performance, despite the fact that the performance of the individual detectors is still often quite modest.

show abstract

Semantic concept-based query expansion and re-ranking for multimedia retrieval

Cited by 161 publications

References 23 publications

Knowledge based query expansion in complex multimedia event detection

Knowledge based query expansion in complex multimedia event detection

Image Retrieval Using Eigen Queries

Improving Automatic Video Retrieval with Semantic Concept Detection

Contact Info

Product

Resources

About