Toward video semantic search based on a structured folksonomy

Abstract

This study investigated the effectiveness of query expansion using synonymous and co-occurrence tags in users' video searches as well as the effect of visual storyboard surrogates on users' relevance judgments when browsing videos. To do so, we designed a structured folksonomy-based system in which tag queries can be expanded via synonyms or co-occurrence words, based on the use of WordNet 2.1 synonyms and Flickr's related tags. To evaluate the structured folksonomy-based system, we conducted an experiment, the results of which suggest that the mean recall rate in the structured folksonomy-based system is statistically higher than that in a tag-based system without query expansion; however, the mean precision rate in the structured folksonomybased system is not statistically higher than that in the tag-based system. Next, we compared the precision rates of the proposed system with storyboards (SB), in which SB and text metadata are shown to users when they browse video search results, with those of the proposed system without SB, in which only text metadata are shown. Our result showed that browsing only text surrogates-including tags without multimedia surrogates-is not sufficient for users' relevance judgments.

Generic speech summarization of transcribed lecture videos: Using tags and their semantic relations

“…In social summarization, video tags known to be similar to or better than indexer‐assigned terms (Heckner, Neubauer, & Wolff, ; Kim, ) can be effectively used as key concepts to select significant sentences from speech transcripts. Heckner et al.…”

“…() claim that videos on YouTube are extensively tagged because uploaders want their videos to be retrieved and viewed by as many people as possible, and not because they want to organize their personal collection. Similarly, Kim () notes that video tags seem to be closer to indexer‐assigned terms than image tags because YouTube videos have a proportionally greater number of tags describing video content than Flickr images.…”

“…Moreover, the function of such a tag method is similar to named‐entity extraction, which uses the frequency of proper names of persons, locations, and organizations. Frequently assigned video tags are related to named entities, such as people and object categories (Kim, ). Named entities, which are often related to topic‐oriented words, are more important in speech summarization than in text summarization (Christensen, Gotoh, Kolluru, & Renalset, ; Zhang, Cohn, & Ciravegna, ).…”

Comparing the use of semantic relations between tags versus latent semantic analysis for generic speech summarization

“…We use Flickr because some words, such as proper nouns and new words that are included in video tags, are not defined in WordNet but can be found in Flickr tags. We assume that our proposed method is closely related to named entity extraction because it was observed that frequently assigned video tags are related to named entities, such as people and object categories (Kim, 2011). Further, we also expect that the semantic relations between tags can be effectively utilized for extracting key sentences from a document.…”

Social speech summarization

“…Moreover, Heckner, Neubauer, and Wolff (2008) stated that videos on YouTube are extensively tagged because users want their videos to be retrieved and viewed by as many people as possible and not because they want to organize their personal collections. Similarly, Kim (2011) suggested that video tags are similar to indexer‐assigned terms and valuable in describing objects or people in a video, since the most frequently employed categories are topics, objects, metadata elements, and people.…”