Modeling Human Judgment of Digital Imagery for Multimedia Retrieval

Volkmer, Timo; Thom, James A.; Tahaghoghi, S. M. M.

doi:10.1109/tmm.2007.900153

Cited by 17 publications

(12 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To cope with the demand for labeled examples, Lin et al initiated a collaborative annotation effort in the TRECVID 2003 benchmark [30]. Using tools from Christel et al [31] and Volkmer et al [32], [33] a common annotation effort was again made for the TRECVID 2005 benchmark, yielding a large and accurate set of labeled examples for 39 concepts taken from a predefined collection [4]. We provided an extension of this compilation, increasing the collection to 101 concept annotations, and also donated the lowlevel features, classifier models, and resulting concept detectors for this set of concepts on TRECVID 2005 and 2006 data as part of the MediaMill Challenge [3].…”

Section: Related Workmentioning

confidence: 99%

Adding Semantics to Detectors for Video Retrieval

Snoek

Huurnink

Hollink

et al. 2007

IEEE Trans. Multimedia

174

140

View full text Add to dashboard Cite

Abstract-In this paper, we propose an automatic video retrieval method based on high-level concept detectors. Research in video analysis has reached the point where over 100 concept detectors can be learned in a generic fashion, albeit with mixed performance. Such a set of detectors is very small still compared to ontologies aiming to capture the full vocabulary a user has. We aim to throw a bridge between the two fields by building a multimedia thesaurus, i.e., a set of machine learned concept detectors that is enriched with semantic descriptions and semantic structure obtained from WordNet. Given a multimodal user query, we identify three strategies to select a relevant detector from this thesaurus, namely: text matching, ontology querying, and semantic visual querying. We evaluate the methods against the automatic search task of the TRECVID 2005 video retrieval benchmark, using a news video archive of 85 h in combination with a thesaurus of 363 machine learned concept detectors. We assess the influence of thesaurus size on video search performance, evaluate and compare the multimodal selection strategies for concept detectors, and finally discuss their combined potential using oracle fusion. The set of queries in the TRECVID 2005 corpus is too small for us to be definite in our conclusions, but the results suggest promising new lines of research.Index Terms-Concept learning, content analysis and indexing, knowledge modeling, multimedia information systems, video retrieval.

show abstract

Section: Related Workmentioning

confidence: 99%

Adding Semantics to Detectors for Video Retrieval

Snoek

Huurnink

Hollink

et al. 2007

IEEE Trans. Multimedia

174

140

View full text Add to dashboard Cite

show abstract

“…We suspect that the level of correctness of our annotations was higher than usual thanks to the special attention the Netherlands Institute for Sound & Vision gave to the collection they prepared for TRECVID. In many cases, of course, human annotators do err and disagree [28]. The timeconsuming nature of human annotation causes the number of subject terms per program to be low, much lower than the number of topics that is visible in the video.…”

Section: Existing Gtaa Relationsmentioning

confidence: 99%

Thesaurus enrichment for query expansion in audiovisual archives

Hollink¹,

Malaisé²,

Schreiber³

2009

Multimed Tools Appl

View full text Add to dashboard Cite

It is common practice in audiovisual archives to disclose documents using metadata from a structured vocabulary or thesaurus. Many of these thesauri have limited or no structure. The objective of this paper is to find out whether retrieval of audiovisual resources from a collection indexed with an in-house thesaurus can be improved by enriching the thesaurus structure. We propose a method to add structure to a thesaurus by anchoring it to an external, semantically richer thesaurus. We investigate the added value of this enrichment for retrieval purposes. We first anchor the thesaurus to an external resource, WordNet. From this anchoring we infer relations between pairs of terms in the thesaurus that were previously unrelated. We employ the enriched thesaurus in a retrieval experiment on a TRECVID 2007 dataset. The results are promising: with simple techniques we are able to enrich a thesaurus in such a way that it adds to retrieval performance.

show abstract

“…We suspect that the level of correctness of our annotations was higher than usual thanks to the special attention the Netherlands Institute for Sound and Vision gave to the collection they prepared for TRECVid. In many cases, of course, human annotators do err and disagree [17]. The time-consuming nature of human annotation causes the number of subject terms per program to be low, much lower than the number of topics that is visible in the video.…”

Section: Existing Gtaa Relationsmentioning

confidence: 99%

Enriching a Thesaurus to Improve Retrieval of Audiovisual Documents

Hollink¹,

Malaisé²,

Schreiber³

2008

Semantic Multimedia

View full text Add to dashboard Cite

Abstract. In many archives of audiovisual documents, retrieval is done using metadata from a structured vocabulary or thesaurus. In practice, many of these thesauri have limited or no structure. The objective of this paper is to find out whether retrieval of audiovisual resources from a collection indexed with an in-house thesaurus can be improved by anchoring the thesaurus to an external, semantically richer thesaurus. We propose a method to enrich the structure of a thesaurus and we investigate its added value for retrieval purposes. We first anchor the thesaurus to an external resource, WordNet. From this anchoring we infer relations between pairs of terms in the thesaurus that were previously unrelated. We employ the enriched thesaurus in a retrieval experiment on a TRECVid 2007 dataset. The results are promising: with simple techniques we are able to enrich a thesaurus in such a way that it adds to retrieval performance.

show abstract

Modeling Human Judgment of Digital Imagery for Multimedia Retrieval

Cited by 17 publications

References 12 publications

Adding Semantics to Detectors for Video Retrieval

Adding Semantics to Detectors for Video Retrieval

Thesaurus enrichment for query expansion in audiovisual archives

Enriching a Thesaurus to Improve Retrieval of Audiovisual Documents

Contact Info

Product

Resources

About