Towards Machine Comprehension of Spoken Content: Initial TOEFL Listening Comprehension Test by Machine

Tseng, Bo-Hsiang; Shen, Sheng-syun; Lee, Hung-yi; Lee, Lin-Shan

doi:10.48550/arxiv.1608.06378

Cited by 7 publications

(12 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Conversational Spoken Answer Type TOEFL (Tseng et al, 2016) × √ Multi-choice Spoken-SQuAD × √ Spans ODSQA × √ Spans Spoken-CoQA √ √ Free-form text, Unanswerable Karakos et al, 2020), translation (Bérard et al, 2016;Di Gangi et al, 2020;Tu et al, 2020) and recognition (Zhang et al, 2017;Bruguier et al, 2019;Siriwardhana et al, 2020). However, simply adopting existing methods to the SCQA tasks will cause several challenges.…”

Section: Datasetmentioning

confidence: 99%

“…It entails transferring spoken content to ASR transcriptions, and then employs natural language processing techniques to handle the speech language processing tasks. Prior to this point, the existing methods (Tseng et al, 2016;Serdyuk et al, 2018;Su & Fung, 2020) focus on optimizing each module in a two-stage manner, where errors in the ASR module would suffer from severe performance loss. Concurrent with our research, Serdyuk et al (2018) proposes an end-to-end approach for natural language understanding (NLU) tasks.…”

Section: Related Workmentioning

confidence: 99%

“…To incorporate spoken content into machine comprehension, there are few public datasets that evaluate the effectiveness of the model in spoken question answering (SQA) scenarios. TOEFL listening comprehension (Tseng et al, 2016) is one of the related corpus for this task, an English test designed to evaluate the English language proficiency of non-native speakers. But the multi-choice question answering setting and its scale is limited to train for robust SCQA models.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Towards Data Distillation for End-to-end Spoken Conversational Question Answering

You,

Chen,

Liu

et al. 2020

Preprint

View full text Add to dashboard Cite

In spoken question answering, QA systems are designed to answer questions from contiguous text spans within the related speech transcripts. However, the most natural way that human seek or test their knowledge is via human conversations. Therefore, we propose a new Spoken Conversational Question Answering task (SCQA), aiming at enabling QA systems to model complex dialogues flow given the speech utterances and text corpora. In this task, our main objective is to build a QA system to deal with conversational questions both in spoken and text forms, and to explore the plausibility of providing more cues in spoken documents with systems in information gathering. To this end, instead of adopting automatically generated speech transcripts with highly noisy data, we propose a novel unified data distillation approach, DDNet, which directly fuse audio-text features to reduce the misalignment between automatic speech recognition hypotheses and the reference transcriptions. In addition, to evaluate the capacity of QA systems in a dialogue-style interaction, we assemble a Spoken Conversational Question Answering (Spoken-CoQA) dataset with more than 120k question-answer pairs. Experiments demonstrate that our proposed method achieves superior performance in spoken conversational question answering.

show abstract

Section: Datasetmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Towards Data Distillation for End-to-end Spoken Conversational Question Answering

You,

Chen,

Liu

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…In spoken question answering (SQA), after transcribing spoken content into text by automatic speech recognition (ASR), typical approaches use information retrieval (IR) techniques [4] to find the proper answer from the ASR hypotheses. One attempt towards QA of spoken content is TOEFL listening comprehension by machine [5]. TOEFL is an English examination that tests the knowledge and skills of academic English for English learners whose native languages are not English.…”

Section: Introductionmentioning

confidence: 99%

Mitigating the Impact of Speech Recognition Errors on Spoken Question Answering by Adversarial Domain Adaptation

Lee

Chen

Lee

2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Spoken question answering (SQA) is challenging due to complex reasoning on top of the spoken documents. The recent studies have also shown the catastrophic impact of automatic speech recognition (ASR) errors on SQA. Therefore, this work proposes to mitigate the ASR errors by aligning the mismatch between ASR hypotheses and their corresponding reference transcriptions. An adversarial model is applied to this domain adaptation task, which forces the model to learn domain-invariant features the QA model can effectively utilize in order to improve the SQA results. The experiments successfully demonstrate the effectiveness of our proposed model, and the results are better than the previous best model by 2% EM score.

show abstract

“…In SQA, after transcribing spoken content into text by automatic speech recognition (ASR), typical approaches use information retrieval (IR) techniques [6] or knowledge bases [7] to find the proper answer from the transcriptions. Another attempt towards machine comprehension of spoken content is TOEFL listening comprehension by machine [8]. TOEFL is an English examination that tests the knowledge and skills of academic English for English learners whose native languages are not English.…”

Section: Introductionmentioning

confidence: 99%

ODSQA: Open-Domain Spoken Question Answering Dataset

Lee

Wang

Chang

et al. 2018

2018 IEEE Spoken Language Technology Workshop (SLT)

Self Cite

View full text Add to dashboard Cite

Reading comprehension by machine has been widely studied, but machine comprehension of spoken content is still a less investigated problem. In this paper, we release Open-Domain Spoken Question Answering Dataset (ODSQA) with more than three thousand questions. To the best of our knowledge, this is the largest real SQA dataset. On this dataset, we found that ASR errors have catastrophic impact on SQA. To mitigate the effect of ASR errors, subword units are involved, which brings consistent improvements over all the models. We further found that data augmentation on text-based QA training examples can improve SQA.

show abstract

Towards Machine Comprehension of Spoken Content: Initial TOEFL Listening Comprehension Test by Machine

Cited by 7 publications

References 0 publications

Towards Data Distillation for End-to-end Spoken Conversational Question Answering

Towards Data Distillation for End-to-end Spoken Conversational Question Answering

Mitigating the Impact of Speech Recognition Errors on Spoken Question Answering by Adversarial Domain Adaptation

ODSQA: Open-Domain Spoken Question Answering Dataset

Contact Info

Product

Resources

About