On Influential Trends in Interactive Video Retrieval: Video Browser Showdown 2015–2017

Lokoč, Jakub; Bailer, Werner; Schoeffmann, Klaus; Muenzer, Bernd; Awad, George

doi:10.1109/tmm.2018.2830110

Cited by 93 publications

(46 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The most popular flavour of interactive learning is user relevance feedback that presents the user, in each interaction round, with the items for which the classification model is most confident [36]. User relevance feedback has frequently been used in the best performing entries of benchmarks focusing on interactive video search and exploration [28,41]. However, those solutions were designed for collections far smaller than YFCC100M, which is the challenge we take in this paper.…”

Section: Related Workmentioning

confidence: 99%

Interactive Learning for Multimedia at Large

Khan

Jónsson

Rudinac

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Interactive learning has been suggested as a key method for addressing analytic multimedia tasks arising in several domains. Until recently, however, methods to maintain interactive performance at the scale of today's media collections have not been addressed. We propose an interactive learning approach that builds on and extends the state of the art in user relevance feedback systems and high-dimensional indexing for multimedia. We report on a detailed experimental study using the ImageNet and YFCC100M collections, containing 14 million and 100 million images respectively. The proposed approach outperforms the relevant state-of-the-art approaches in terms of interactive performance, while improving suggestion relevance in some cases. In particular, even on YFCC100M, our approach requires less than 0.3 s per interaction round to generate suggestions, using a single computing core and less than 7 GB of main memory.

show abstract

Section: Related Workmentioning

confidence: 99%

Interactive Learning for Multimedia at Large

Khan

Jónsson

Rudinac

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…The second part of the tutorial is intended as a practical exercise in the form of a lightweight installment of the Video Browser Showdown competition [6,7] for the tutorial participants. Following the success of novice user sessions from the Video Browser Showdown, we believe that the participants quickly gain an experience with the topic, tools, tasks and related problems.…”

Section: Live Evaluationmentioning

confidence: 99%

“…The third part will provide an overview of existing evaluation campaigns, such as the VBS [6,7], the LSC [3] or TRECVID [1], outline their tasks, goals, commonalities and differences and discuss their evaluation strategies. The choice of evaluation strategies is not only influenced by aspects such as repeatability and the reuse of assessments, but also impacted by the setting of the evaluation campaign, i.e., whether the competition is live in front of the audience (as e.g.…”

Section: Evaluation Campaignsmentioning

confidence: 99%

Interactive Video Retrieval in the Age of Deep Learning

Lokoč

Schoeffmann

Bailer

et al. 2019

Proceedings of the 2019 on International Conference on Multimedia Retrieval

Self Cite

View full text Add to dashboard Cite

We present a tutorial focusing on video retrieval tasks, where stateof-the-art deep learning approaches still benefit from interactive decisions of users. The tutorial covers general introduction to the interactive video retrieval research area, state-of-the-art video retrieval systems, evaluation campaigns and recently observed results. Moreover, a significant part of the tutorial is dedicated to a practical exercise with three selected state-of-the-art systems in the form of an interactive video retrieval competition. Participants of this tutorial will gain a practical experience and also a general insight of the interactive video retrieval topic, which is a good start to focus their research on unsolved challenges in this area.

show abstract

“…Blackthorn, for example, compresses semantic information from the visual and text domain and learns user preferences on the fly from the interactions with the system in a relevance feedback framework. Serving as the epicenter of research on interactive multimedia retrieval, the initiatives such as Video Browser Showdown produced a number of excellent analytics systems [13]. [24] GraphViz [9] PIWI [35] Newdle [36] Gephi [2] CoMeRDA [5] Blackthorn [38] vitrivr [20] SIRET [12] Vibro [1] PICTuReVis [29] ISOLDE For example, vitrivr system owes it good performance in interactive multimedia retrieval to an indexing structure for efficient kNN search [20].…”

Section: Multimedia Analyticsmentioning

confidence: 99%

Interactive Search and Exploration in Discussion Forums Using Multimodal Embeddings

Gornishka

Rudinac

Worring

2019

MultiMedia Modeling

View full text Add to dashboard Cite

In this paper we present a novel interactive multimodal learning system, which facilitates search and exploration in large networks of social multimedia users. It allows the analyst to identify and select users of interest, and to find similar users in an interactive learning setting. Our approach is based on novel multimodal representations of users, words and concepts, which we simultaneously learn by deploying a general-purpose neural embedding model. We show these representations to be useful not only for categorizing users, but also for automatically generating user and community profiles. Inspired by traditional summarization approaches, we create the profiles by selecting diverse and representative content from all available modalities, i.e. the text, image and user modality. The usefulness of the approach is evaluated using artificial actors, which simulate user behavior in a relevance feedback scenario. Multiple experiments were conducted in order to evaluate the quality of our multimodal representations, to compare different embedding strategies, and to determine the importance of different modalities. We demonstrate the capabilities of the proposed approach on two different multimedia collections originating from the violent online extremism forum Stormfront and the microblogging platform Twitter, which are particularly interesting due to the high semantic level of the discussions they feature. CCS CONCEPTS• Information systems → Multimedia and multimodal retrieval. KEYWORDS multimedia analytics, search, exploration, interactive learning, multimodal embeddings, online discussion forums, social multimedia • First, compact but meaningful multimodal content representations are needed to ensure the interactivity of the system

show abstract

On Influential Trends in Interactive Video Retrieval: Video Browser Showdown 2015–2017

Cited by 93 publications

References 42 publications

Interactive Learning for Multimedia at Large

Interactive Learning for Multimedia at Large

Interactive Video Retrieval in the Age of Deep Learning

Interactive Search and Exploration in Discussion Forums Using Multimodal Embeddings

Contact Info

Product

Resources

About