Automated Metadata in Multimedia Information Systems: Creation, Refinement, Use in Surrogates, and Evaluation

Abstract

Improvements in network bandwidth along with dramatic drops in digital storage and processing costs have resulted in the explosive growth of multimedia (combinations of text, image, audio, and video) resources on the Internet and in digital repositories. A suite of computer technologies delivering speech, image, and natural language understanding can automatically derive descriptive metadata for such resources. Difficulties for end users ensue, however, with the tremendous volume and varying quality of automated metadata for multimedia information systems. This lecture surveys automatic metadata creation methods for dealing with multimedia information resources, using broadcast news, documentaries, and oral histories as examples. Strategies for improving the utility of such metadata are discussed, including computationally intensive approaches, leveraging multimodal redundancy, folding in context, and leaving precision-recall tradeoffs under user control. Interfaces building from automatically generated metadata are presented, illustrating the use of video surrogates in multimedia information systems. Traditional information retrieval evaluation is discussed through the annual National Institute of Standards and Technology TRECVID forum, with experiments on exploratory search extending the discussion beyond fact-finding to broader, longer term search activities of learning, analysis, synthesis, and discovery.

An examination of metadata practices for research data reuse: Characteristics and predictive probability of metadata elements

“…Among a wide variety of influencing factors, metadata is recognized as the foundation of data preservation and discovery to support data reuse (Christel 2009;Gartner 2016;Qin, Ball and Greenberg 2012;Star and Gastl 2011) as well as is central to the data citation framework (Li, Greenberg and Lin 2016). Well-formed and right-sized metadata is recognized as a critical element of a working infrastructure capable of meeting the needs of data access, identification, and reuse (Star and Gastl 2011) .…”

Multimedia Pivot Tables for Multimedia Analytics on Image Collections

“…One of the first systems doing so is the Informedia system which employs speech and video analysis in conjunction with effective user interfaces [7]. A system targeting computer vision algorithm developers is presented in [16] allowing the user to gain insight in features and how to use these features in surveillance.…”

Multimedia

“…However, integrating across multiple techniques and bringing in an intelligent human user to review and direct investigations can sidestep the noise and result in improved information precision. A series of experiments, including those conducted under the NIST TRECVID annual series, are surveyed in [1] and will be introduced to start off the multimedia keynote discussion. Video especially is a very rich information medium, but looking through thousands of hours of video to find specific examples is impractical; through automated techniques, that task becomes feasible.…”