BVideoQA: Online English/Chinese bilingual video question answering

Abstract

This article presents a bilingual video question answering (QA) system, namely BVideoQA, which allows users to retrieve Chinese videos through English or Chinese natural language questions. Our method first extracts an optimal one-to-one string pattern matching according to the proposed dense and long N -gram match. On the basis of the matched string patterns, it gives a passage score based on our term-weighting scheme. The main contributions of this approach to multimedia information retrieval literatures include: (a) development of a truly bilingual video QA system, (b) presentation of a robust bilingual passage retrieval algorithm to handle no-wordboundary languages such as Chinese and Japanese, (c) development of a large-scale bilingual video QA corpus for system evaluation, and (d) comparisons of seven top-performing retrieval methods under the fair conditions. The experimental studies indicate that our method is superior to other existing approaches in terms of precision and main rank reciprocal rates. When ported to English, encouraging empirical results also are obtained. Our method is very important to Asian-like languages since the development of a word tokenizer is optional.

Investigation of learners' perceptions for video summarization and recommendation

“…Summarization is handled by the mechanism of question and answering (Q/A based approach). More specifically, learners can submit their questions and then the system will find appropriate answers for the questions by extracting passage-level answers from different segments of video captions (Lee, Wu, & Yang, 2009). These extracted answers are used to generate video summaries by using a ranking algorithm (Wu, Yang, & Lee, 2008).…”

A Study of Video-based Concordancer on Scene Classification

“…Both W_density(Q,C i ) and W_weight(Q,C i ) are two different weighting schemes for query Q and subset C i (details can be found in [19]). K denotes as the number of sentences in scene SC.…”

“…This provides flexible interface for search. The BVideoQA [19] are employed as the full text retrieval engine which showed more accurate result than the other famous retrieval algorithms 1 . By means of scene tagging and segmenting, each scene with its associated caption words is treated as a paragraph and the input to the BVideoQA module.…”