Interactive Search or Sequential Browsing? A Detailed Analysis of the Video Browser Showdown 2018

Abstract

This work summarizes the findings of the 7th iteration of the Video Browser Showdown (VBS) competition organized as a workshop at the 24th International Conference on Multimedia Modeling in Bangkok. The competition focuses on video retrieval scenarios in which the searched scenes were either previously observed or described by another person (i.e., an example shot is not available). During the event, nine teams competed with their video retrieval tools in providing access to a shared video collection with 600 hours of video content. Evaluation objectives, rules, scoring, tasks, and all participating tools are described in the article. In addition, we provide some insights into how the different teams interacted with their video browsers, which was made possible by a novel interaction logging mechanism introduced for this iteration of the VBS. The results collected at the VBS evaluation server confirm that searching for one particular scene in the collection when given a limited time is still a challenging task for many of the approaches that were showcased during the event. Given only a short textual description, finding the correct scene is even harder. In ad hoc search with multiple relevant scenes, the tools were mostly able to find at least one scene, whereas recall was the issue for many teams. The logs also reveal that even though recent exciting advances in machine learning narrow the classical semantic gap problem, user-centric interfaces are still required to mediate access to specific content. Finally, open challenges and lessons learned are presented for future VBS events.

Interactive Video Retrieval in the Age of Deep Learning

“…The last part of the tutorial focuses on observed results at the Video Browser Showdown [6,7] and Lifelog Search Challenge [3]. For both competitions, we present evaluated tasks, metrics, scoring and constraints.…”

“…The second part of the tutorial is intended as a practical exercise in the form of a lightweight installment of the Video Browser Showdown competition [6,7] for the tutorial participants. Following the success of novice user sessions from the Video Browser Showdown, we believe that the participants quickly gain an experience with the topic, tools, tasks and related problems.…”

“…The third part will provide an overview of existing evaluation campaigns, such as the VBS [6,7], the LSC [3] or TRECVID [1], outline their tasks, goals, commonalities and differences and discuss their evaluation strategies. The choice of evaluation strategies is not only influenced by aspects such as repeatability and the reuse of assessments, but also impacted by the setting of the evaluation campaign, i.e., whether the competition is live in front of the audience (as e.g.…”

VERGE in VBS 2020

“…VERGE is an interactive video retrieval system that provides users with efficient browsing and various search capabilities inside a set of video collections. For more than ten years, VERGE has been participating in numerous video retrieval related conferences and showcases, including TRECVID [1] and Video Browser Showdown (VBS) [2], thus the system is adapted to support the Known Item Search (KIS), Instance Search (INS) and Ad-Hoc Video Search (AVS) tasks. Experience from previous participation drove this year's selection of mature solutions (Section 2.1), the improvement of old modalities (Sections 2.2, 2.6), the integration of new (Sections 2.3, 2.4, 2.5) and also any advances regarding the user experience.…”

SOMHunter: Lightweight Video Search System with SOM-Guided Relevance Feedback

“…known-item search system [8,9]. However, many research teams in the multimedia retrieval community focus just on a specific category of tasks, and find it too time-demanding to include their developed model in a more complex interactive video search framework.…”