Towards Machine Comprehension of Spoken Content: Initial TOEFL Listening Comprehension Test by Machine

Abstract

Multimedia or spoken content presents more attractive information than plain text content, but it's more difficult to display on a screen and be selected by a user. As a result, accessing large collections of the former is much more difficult and time-consuming than the latter for humans. It's highly attractive to develop a machine which can automatically understand spoken content and summarize the key information for humans to browse over. In this endeavor, we propose a new task of machine comprehension of spoken content. We define the initial goal as the listening comprehension test of TOEFL, a challenging academic English examination for English learners whose native language is not English. We further propose an Attention-based Multi-hop Recurrent Neural Network (AM-RNN) architecture for this task, achieving encouraging results in the initial tests. Initial results also have shown that word-level attention is probably more robust than sentence-level attention for this task with ASR errors.

Mitigating the Impact of Speech Recognition Errors on Spoken Question Answering by Adversarial Domain Adaptation

“…In spoken question answering (SQA), after transcribing spoken content into text by automatic speech recognition (ASR), typical approaches use information retrieval (IR) techniques [4] to find the proper answer from the ASR hypotheses. One attempt towards QA of spoken content is TOEFL listening comprehension by machine [5]. TOEFL is an English examination that tests the knowledge and skills of academic English for English learners whose native languages are not English.…”

ODSQA: Open-Domain Spoken Question Answering Dataset

“…In SQA, after transcribing spoken content into text by automatic speech recognition (ASR), typical approaches use information retrieval (IR) techniques [6] or knowledge bases [7] to find the proper answer from the transcriptions. Another attempt towards machine comprehension of spoken content is TOEFL listening comprehension by machine [8]. TOEFL is an English examination that tests the knowledge and skills of academic English for English learners whose native languages are not English.…”

“…TOEFL is an English examination that tests the knowledge and skills of academic English for English learners whose native languages are not English. Deep-based models including attention-based RNN [8] and tree-structured RNN [9] were Thanks to Delta Research Center and Delta Electronics, Inc. for collecting the DRCD dataset. used to answer TOEFL listening comprehension test.…”

YNU Deep at SemEval-2018 Task 12: A BiLSTM Model with Neural Attention for Argument Reasoning Comprehension

“…This allows a model to focus on the aspects of a document that it believes helpful to answer a question. The attention-based LSTM models have achieved state-of-the-art results in machine comprehension tasks (Kadlec et al, 2016;Chen et al, 2016;Tseng et al, 2016).…”

“…Compared to traditional machine comprehension task, argument reasoning comprehension requires models to possess extra reasoning abilities. Some models increase the depth of the network, continuously updating the representations of the documents and questions to realize the reasoning process (Sukhbaatar et al, 2015;Tseng et al, 2016;Dhingra et al, 2017;Sordoni et al, 2016).…”