Interactive Video Retrieval in the Age of Deep Learning

Abstract

We present a tutorial focusing on video retrieval tasks, where stateof-the-art deep learning approaches still benefit from interactive decisions of users. The tutorial covers general introduction to the interactive video retrieval research area, state-of-the-art video retrieval systems, evaluation campaigns and recently observed results. Moreover, a significant part of the tutorial is dedicated to a practical exercise with three selected state-of-the-art systems in the form of an interactive video retrieval competition. Participants of this tutorial will gain a practical experience and also a general insight of the interactive video retrieval topic, which is a good start to focus their research on unsolved challenges in this area.

Evaluating Performance and Trends in Interactive Video Retrieval: Insights From the 12th VBS Competition

“…Each cell in the table represents the fraction #agreement #disagreement , where #agreement and #disagreement represent the number of identical submitted shots judged as correct and wrong, respectively. For instance, in task a2, the red cell showing [70], significant disagreement appears in more tasks, i.e., task a2 and a3. In task a2, seven teams disagreed with the judgment on one video, while four teams disagreed on one video in task a3.…”

“…Teams receive scores for the first correct submission of each video, and a penalty is added for wrong submissions to prevent the submission of unverified shots. The score f t of a team t is determined as in [70]:…”

An overview of video recommender systems: state-of-the-art and research issues

“…Those models are able to accurately predict labels with few (few-shot) or none (zero-shot) labeled examples. While these models already have been shown to outperform other approaches in interactive video retrieval (Lokoč et al, 2023), their potential in video recommendation remains largely unexplored. Future research could focus on applications in recommendation systems where historical interaction data is limited or absent, potentially improving cold start scenarios.…”

Improving Query and Assessment Quality in Text-Based Interactive Video Retrieval Evaluation

“…Based on data analysis from two benchmark editions (2021 and 2022) published in [3,4], we perform an analysis of the agreement between assessors and the participating teams. In 2021, only a discussion of queries was performed, while in 2022, the process described in this paper was introduced.…”