Examining user interactions with video retrieval systems

Abstract

The Informedia group at Carnegie Mellon University has since 1994 been developing and evaluating surrogates, summary interfaces, and visualizations for accessing digital video collections containing thousands of documents, millions of shots, and terabytes of data. This paper reports on TRECVID 2005 and 2006 interactive search tasks conducted with the Informedia system by users having no knowledge of Informedia or other video retrieval interfaces, but being experts in analyst activities. Think-aloud protocols, questionnaires, and interviews were also conducted with this user group to assess the contributions of various video summarization and browsing techniques with respect to broadcast news test corpora. Lessons learned from these user interactions are reported, with recommendations on both interface improvements for video retrieval systems and enhancing the ecological validity of video retrieval interface evaluations.

Search strategies in multimodal image retrieval

“…Understanding preferred user tactics for different situations enables better support for searchers. While contentbased retrieval methods have limited utility on their own, having multiple access strategies to visual content has been found beneficial [Christel 2007]. Also offered should be support for multimodal query formulation.…”

Visual information seeking

“…Christel () measured the effectiveness of video summarization and browsing using the TRECVid protocol for postconference analysis. The common data set (video collection) and search topic test set from two previous years of TRECVid were employed across different user experiments.…”

“…These users had no prior experience or expertise with the experimental system or even with video retrieval in general. On top of the structured TRECVid search experiments, Christel () also conducted exploratory search experiments, noting that users preferred and became motivated with finding information that is of interest to them, as opposed to random search topics pertinent from others, such as TRECVid. It was then noted that an important aspect of exploratory search, relevant to user‐centered evaluations, was that system performance measures of relevance were not always as significant in this context, which is noteworthy for reviewing methods and approaches for interactive video retrieval and informs experimental designs.…”

“…Christel () complemented a set of experiments of professional situation analysts with qualitative methods to examine exploratory search within an interactive video retrieval context. Interview and think‐aloud data were gathered, on top of a TRECVid search task protocol, providing additional insight into the quantitative data and results.…”

An experimental evaluation of ontology-based user profiles

“…Christel [9] critisises that in the interactive video retrieval domain, most research approaches focus on short-term retrieval as advocated within the TRECVid evaluation campaign, hence ignoring more realistic video retrieval scenarios. Considering the broader focus of realistic video search, they argue for "Multi-dimensional In-depth Long-Term Case studies" (MILC), as advertised by Shneiderman and Plaisant [24].…”