Dataset and Neural Recurrent Sequence Labeling Model for Open-Domain Factoid Question Answering

Li, Wei; He, Zhengyan; Wang, Xuguang; Cao, Ying; Zhou, Jie; Xu, Wei

doi:10.48550/arxiv.1607.06275

Cited by 38 publications

(36 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The answer extraction task is intended to extract the answer word from the answer sentence (RQ3). These tasks are crucial in the study of machine reading comprehension [31,51,55]. Note that we aim to demonstrate the effectiveness and interpretability of EEG signals as implicit feedback.…”

Section: Experiments and Discussionmentioning

confidence: 99%

Understanding Human Reading Comprehension with Brain Signals

Ye,

Xie,

Liu

et al. 2021

Preprint

View full text Add to dashboard Cite

Reading comprehension is a complex cognitive process involving many human brain activities. Plenty of works have studied the reading patterns and attention allocation mechanisms in the reading process. However, little is known about what happens in human brain during reading comprehension and how we can utilize this information as implicit feedback to facilitate information acquisition performance. With the advances in brain imaging techniques such as electroencephalogram (EEG), it is possible to collect highprecision brain signals in almost real time.With neuroimaging techniques, we carefully design a lab-based user study to investigate brain activities during reading comprehension. Our findings show that neural responses vary with different types of contents, i.e., contents that can satisfy users' information needs and contents that cannot. We suggest that various cognitive activities, e.g., cognitive loading, semantic-thematic understanding, and inferential processing, at the micro-time scale during reading comprehension underpin these neural responses. Inspired by these detectable differences in cognitive activities, we construct supervised learning models based on EEG features for two reading comprehension tasks: answer sentence classification and answer extraction. Results show that it is feasible to improve their performance with brain signals. These findings imply that brain signals are valuable feedback for enhancing human-computer interactions during reading comprehension.

show abstract

Section: Experiments and Discussionmentioning

confidence: 99%

Understanding Human Reading Comprehension with Brain Signals

Ye,

Xie,

Liu

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Closed-book question answering (QA) tasks, including WebQA [38]. We follow the same closed-book setting in GPT-3 [1], where the models are not allowed to access any external knowledge when answering open-domain factoid questions about broad factual knowledge.…”

Section: Task Descriptionmentioning

confidence: 99%

PanGu-$α$: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation

Zeng,

Ren,

et al. 2021

Preprint

View full text Add to dashboard Cite

Large-scale Pretrained Language Models (PLMs) have become the new paradigm for Natural Language Processing (NLP). PLMs with hundreds of billions parameters such as have demonstrated strong performances on natural language understanding and generation with few-shot in-context learning. In this work, we present our practice on training large-scale autoregressive language models named PanGu-α, with up to 200 billion parameters. PanGu-α is developed under the MindSpore 2 and trained on a cluster of 2048 Ascend 910 AI processors 3 . The training parallelism strategy is implemented based on MindSpore Auto-parallel, which composes five parallelism dimensions to scale the training task to 2048 processors efficiently, including data parallelism, op-level model parallelism, pipeline model parallelism, optimizer model parallelism and rematerialization. To enhance the generalization ability of PanGu-α, we collect 1.1TB high-quality Chinese data from a wide range of domains to pretrain the model. We empirically test the generation ability of PanGu-α in various scenarios including text summarization, question answering, dialogue generation, etc. Moreover, we investigate the effect of model scales on the few-shot performances across a broad range of Chinese NLP tasks. The experimental results demonstrate the superior capabilities of PanGu-α in performing various tasks under few-shot or zero-shot settings.

show abstract

“…When answer type is multi-span, ms represents the sequence labels of this answer, otherwise null. We adopt the B, I, O scheme to indicate multi-span answer (Li et al, 2016) in which ms = (n 1 , . .…”

Section: Mrc Modelmentioning

confidence: 99%

No Answer is Better Than Wrong Answer: A Reflection Model for Document Level Machine Reading Comprehension

Wang

Shou

Gong

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

The Natural Questions (NQ) benchmark set brings new challenges to Machine Reading Comprehension: the answers are not only at different levels of granularity (long and short), but also of richer types (including no-answer, yes/no, single-span and multi-span). In this paper, we target at this challenge and handle all answer types systematically. In particular, we propose a novel approach called Reflection Net which leverages a two-step training procedure to identify the no-answer and wrong-answer cases. Extensive experiments are conducted to verify the effectiveness of our approach. At the time of paper writing (May. 20, 2020), our approach achieved the top 1 on both long and short answer leaderboard * , with F1 scores of 77.2 and 64.1, respectively.

show abstract

Dataset and Neural Recurrent Sequence Labeling Model for Open-Domain Factoid Question Answering

Cited by 38 publications

References 11 publications

Understanding Human Reading Comprehension with Brain Signals

Understanding Human Reading Comprehension with Brain Signals

PanGu-$α$: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation

No Answer is Better Than Wrong Answer: A Reflection Model for Document Level Machine Reading Comprehension

Contact Info

Product

Resources

About