A Robust Passage Retrieval Algorithm for Video Question Answering

Abstract

In this paper, we present a robust passage retrieval algorithm to extend the conventional text question answering (Q/A) to videos. Users interact with our videoQ/A system through natural language queries, while the top-ranked passage fragments with associated video clips are returned as answers. We compare our method with five of the high-performance ranking algorithms that are portable to different languages and domains. The experiments were evaluated with 75.3 h of Chinese videos and 253 questions. The experimental results showed that our method outperformed the second best retrieval model (language models) in relatively 1.43% in mean reciprocal rank (MRR) score and 11.36% when employing a Chinese word segmentation tool. By adopting the initial retrieval results from the retrieval models, our method yields an improvement of at least 5.94% improvement in MRR score. This makes it very attractive for the Asia-like languages since the use of a well-developed word tokenizer is unnecessary. Index Terms-Multimedia retrieval, question answering (Q/A), video question answering (videoQ/A).

Comparative Study of Multimedia Question Answering System Models

“…Some video frameworks related to Question Answering mechanism were proposed and large part of them depended on the utilization of text records received from video oriented representation by person with proper acknowledgment and programmed discourse acknowledgment yields. ( 8) - (12). Tom Yeh, John J. Lee and Trevor Darrell were the principal picture based QA (13).…”