VideOlympics: Real-Time Evaluation of Multimedia Retrieval Systems

Snoek, Cees G. M.; Worring, Marcel; Rooij, O. de; Sande, Koen E. A. van de; Yan, Rong; Hauptmann, Alex

doi:10.1109/mmul.2008.21

Cited by 45 publications

(23 citation statements)

References 1 publication

(1 reference statement)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The aim is to evaluate video browsing tools for their efficiency at KIS tasks with a well-defined data set and setup in direct comparison to other tools. It is important to note that the VBS shares some similarities with the VideOlympics [34] (such as running known-item queries in front of a live audience) but differs in terms of rules and targeted methods/tools. While the VideOlympics mainly focuses on automatic video retrieval in video collections, the VBS targets highly interactive search tools that support users in exploiting their knowledge about the content for faster visual search.…”

Section: Introductionmentioning

confidence: 99%

The Video Browser Showdown: a live evaluation of interactive video search tools

Schoeffmann

Ahlström

Bailer

et al. 2013

Int J Multimed Info Retr

View full text Add to dashboard Cite

The Video Browser Showdown evaluates the performance of exploratory video search tools on a common data set in a common environment and in presence of the audience. The main goal of this competition is to enable researchers in the field of interactive video search to directly compare their tools at work. In this paper we present results from the second Video Browser Showdown (VBS2013) and describe and evaluate the tools of all participating teams in detail. The evaluation results give insights on how exploratory video search tools are used and how they perform in direct compari- son. Moreover, we compare the achieved performance to results from another user study where 16 participants employed a standard video player to complete the same tasks as performed in VBS2013. This comparison shows that the sophisticated tools enable better performance in general but for some tasks common video players provide similar performance and could even outperform the expert tools. Our results highlight the need for further improvement of professional tools for interactive search in videos.

show abstract

Section: Introductionmentioning

confidence: 99%

The Video Browser Showdown: a live evaluation of interactive video search tools

Schoeffmann

Ahlström

Bailer

et al. 2013

Int J Multimed Info Retr

View full text Add to dashboard Cite

show abstract

“…Visualization tools can also be successfully used to help semantic video search (see for example the MediaMill semantic video search engine, [21,22]). …”

Section: Related Workmentioning

confidence: 99%

Interactive visualization of video content and associated description for semantic annotation

Campanella

Leonardi

Migliorati

2008

SIViP

View full text Add to dashboard Cite

In this paper, we present an intuitive graphic framework introduced for the effective visualization of video content and associated audio-visual description, with the aim to facilitate a quick understanding and annotation of the semantic content of a video sequence. The basic idea consists in the visualization of a 2D feature space in which the shots of the considered video sequence are located. Moreover, the temporal position and the specific content of each shot can be displayed and analysed in more detail. The selected features are decided by the user, and can be updated during the navigation session. In the main window, shots of the considered video sequence are displayed in a Cartesian plane, and the proposed environment offers various functionalities for automatically and semi-automatically finding and annotating the shot clusters in such feature space. With this tool the user can therefore explore graphically how the basic segments of a video sequence are distributed in the feature space, and can recognize and annotate the significant clusters and their structure. The experimental results show that browsing and annotating documents with the aid of the proposed visualization paradigms is easy and quick, since the user has a fast and intuitive access to the audio-video content, even if he or she has not seen the document yet.

show abstract

“…For example, the VideOlympics [1] is an annual event in the multimedia research community to showcase and promote various outcomes of video retrieval research, where the researchers bring to the event their interactive systems and demonstrators that incorporate cutting-edge video retrieval techniques and algorithms. These demonstrators highlight interactivity in which an end user might engage to search, browse, and be entertained by video content in ways that have not been tried before.…”

Section: Introductionmentioning

confidence: 99%

Designing Interactive Applications to Support Novel Activities

Lee

Ali

Hardman

2013

Advances in Human-Computer Interaction

View full text Add to dashboard Cite

R&D in media-related technologies including multimedia, information retrieval, computer vision, and the semantic web is experimenting on a variety of computational tools that, if sufficiently matured, could support many novel activities that are not practiced today. Interactive technology demonstration systems produced typically at the end of their projects show great potential for taking advantage of technological possibilities. These demo systems or "demonstrators" are, even if crude or farfetched, a significant manifestation of the technologists' visions in transforming emerging technologies into novel usage scenarios and applications. In this paper, we reflect on design processes and crucial design decisions made while designing some successful, web-based interactive demonstrators developed by the authors. We identify methodological issues in applying today's requirementdriven usability engineering method to designing this type of novel applications and solicit a clearer distinction between designing mainstream applications and designing novel applications. More solution-oriented approaches leveraging design thinking are required, and more pragmatic evaluation criteria is needed that assess the role of the system in exploiting the technological possibilities to provoke further brainstorming and discussion. Such an approach will support a more efficient channelling of the technology-to-application transformation which are becoming increasingly crucial in today's context of rich technological possibilities.

show abstract

VideOlympics: Real-Time Evaluation of Multimedia Retrieval Systems

Cited by 45 publications

References 1 publication

The Video Browser Showdown: a live evaluation of interactive video search tools

The Video Browser Showdown: a live evaluation of interactive video search tools

Interactive visualization of video content and associated description for semantic annotation

Designing Interactive Applications to Support Novel Activities

Contact Info

Product

Resources

About