The trecvid 2008 BBC rushes summarization evaluation

Abstract

This paper describes an evaluation of automatic video summarization systems run on rushes from several BBC dramatic series. It was carried out under the auspices of the TREC Video Retrieval Evaluation (TRECVid) as a followup to the 2007 video summarization workshop held at ACM Multimedia 2007. 31 research teams submitted video summaries of 40 individual rushes video files, aiming to compress out redundant and insignificant material. Each summary had a duration of at most 2% of the original. The output of a baseline system, which simply presented each full video at 50 times normal speed was contributed by Carnegie Mellon University (CMU) as a control.The 2007 procedures for developing ground truth lists of important segments from each video were applied at the National Institute of Standards and Technology (NIST) to the BBC videos. At Dublin City University (DCU) each summary was judged by 3 humans with respect to how much of the ground truth was included and how well-formed the summary was. Additional objective measures included: how long it took the system to create the summary, how long it took the assessor to judge it against the ground truth, and what the summary's duration was. Assessor agreement on finding desired segments averaged 81%. Results indicated that while it was still difficult to exceed the performance of the baseline on in-1 cluding ground truth, the baseline was outperformed by most other systems with respect to avoiding redundancy/junk and presenting the summary with a pleasant tempo/rhythm.

Towards key-frame extraction methods for 3D video: a review

“…In the past evaluation frameworks for 2D key-frame summarisation methods were proposed in [90,91]. More recently, Avila et al [24] also proposed another evaluation setup, wherein the original video and the key-frame summaries of several methods are available for downloading, together with the results of several key-frame extraction methods for 2D video.…”

YouTube Scale, Large Vocabulary Video Annotation

“…The authors of [14] also argue that their underlying bernoulli model for annotations is more appropriate for image keyword annotations where words are not repeated compared to the multinomial assumptions used in their earlier work [22]. The experimental analysis of the multiple bernoulli model of [14] used a subset of the NIST Video Trec dataset [34]. Their dataset consisted of 12 MPEG files, each 30 minutes long from CNN or ABC including advertisements.…”

“…For these reasons we briefly review some relevant elements of TRECVID here and discuss some recent observations and developments. More details about the 2008 competition are given in [34].…”

Rethinking Summarization and Storytelling for Modern Social Multimedia

“…100-word) summary of a set of news articles. TRECVID BBC Rushes summarization was probably the first systematic e↵ort in the multimedia and computer vision communities focusing on video summarization [35]. The task involved reducing a raw and unstructured video captured during the recording of a TV series to a short segment of just a couple of minutes.…”