Multi-Scale Attentive Interaction Networks for Chinese Medical Question Answer Selection

Zhang, Sheng; Zhang, Xin; Wang, Hui; Guo, Lixiang; Liu, Shanshan

doi:10.1109/access.2018.2883637

Cited by 30 publications

(22 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Closed-Book Question Answering (CB-QA): NLPCC-DBQA 16 , CHIP2019, cMedQA [78], cMedQA2 [79], CKBQA 17 , WebQA [80].…”

Section: Evaluation Tasksmentioning

confidence: 99%

ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation

Wang¹,

Sun²,

Xiang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Pre-trained language models have achieved state-of-the-art results in various Natural Language Processing (NLP) tasks. has shown that scaling up pre-trained language models can further exploit their enormous potential. A unified framework named ERNIE 3.0 [2] was recently proposed for pre-training large-scale knowledge enhanced models and trained a model with 10 billion parameters. ERNIE 3.0 outperformed the state-of-the-art models on various NLP tasks. In order to explore the performance of scaling up ERNIE 3.0, we train a hundred-billion-parameter model called ERNIE 3.0 Titan with up to 260 billion parameters on the PaddlePaddle [3] platform. Furthermore, we design a self-supervised adversarial loss and a controllable language modeling loss to make ERNIE 3.0 Titan generate credible and controllable texts. To reduce the computation overhead and carbon emission, we propose an online distillation framework for ERNIE 3.0 Titan, where the teacher model will teach students and train itself simultaneously. ERNIE 3.0 Titan is the largest Chinese dense pre-trained model so far. Empirical results show that the ERNIE 3.0 Titan outperforms the state-of-the-art models on 68 NLP datasets.

show abstract

“…• Closed-Book Question Answering (CB-QA): NLPCC-DBQA 16 , CHIP2019, cMedQA [78], cMedQA2 [79], CKBQA 17 , WebQA [80].…”

Section: Evaluation Tasksmentioning

confidence: 99%

ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation

Wang¹,

Sun²,

Xiang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…For this, a large collection of possible long answers are collected, and the main objective of the task is to identify the correct answer to a given question [212,211].…”

Section: • Answer Matchingmentioning

confidence: 99%

Recent Advances in Automated Question Answering In Biomedical Domain

Baksi¹

2021

Preprint

View full text Add to dashboard Cite

The objective of automated Question Answering (QA) systems is to provide answers to user queries in a time efficient manner. The answers are usually found in either databases (or knowledge bases) or a collection of documents commonly referred to as the corpus. In the past few decades there has been a proliferation of acquisition of knowledge and consequently there has been an exponential growth in new scientific articles in the field of biomedicine. Therefore, it has become difficult to keep track of all the information in the domain, even for domain experts. With the improvements in commercial search engines, users can type in their queries and get a small set of documents most relevant for answering their query, as well as relevant snippets from the documents in some cases. However, it may be still tedious and time consuming to manually look for the required information or answers. This has necessitated the development of efficient QA systems which aim to find exact and precise answers to user provided natural language questions in the domain of biomedicine. In this paper, we introduce the basic methodologies used for developing general domain QA systems, followed by a thorough investigation of different aspects of biomedical QA systems, including benchmark datasets and several proposed approaches, both using structured databases and collection of texts. We also explore the limitations of current systems and explore potential avenues for further advancement.

show abstract

“…Memory networks as a storage mechanism, whose memory cells are continuous vectors, are efficient in capturing long-term dependencies [40]. The use of memory networks has been beneficial in multiple research areas, such as speech detection [41], [42], question-answering [43], [44], machine translation [45], commonsense reasoning [46], and others. Zadeh et al [47] proposed a framework of memory-based sequential learning for multi-view signals.…”

Section: Related Workmentioning

confidence: 99%

Different Contextual Window Sizes Based RNNs for Multimodal Emotion Detection in Interactive Conversations

Lai

Chen

2020

IEEE Access

View full text Add to dashboard Cite

Multimodal emotion detection (MED) in interactive conversations is extremely important for improving the overall human-computer interaction experience. Present research methods in this domain do not explicitly distinguish the contexts of a test utterance in a meaningful way while classifying emotions in conversations. In this paper, we propose a model, named different contextual window sizes based recurrent neural networks (DCWS-RNNs), to differentiate the contexts. The model has four recurrent neural networks (RNNs) that use different contextual window sizes. These window sizes can represent the implicit weights of different aspects of contexts. Further, four RNNs are independently to model the different aspects of contexts into memories. Such memories with the test utterance are then merged using attention-based multiple hops. Experiments show DCWS-RNNs outperforms the compared methods on both the IEMOCAP and AVEC datasets. Case studies on the IEMOCAP dataset also demonstrate that our model has excellent performance to capture the emotional dependent utterance that is most relevant to the test utterance and assigned to the highest attention score. INDEX TERMS Interactive conversations, contextual window sizes, emotion detection, multimodal, recurrent neural network.

show abstract

Multi-Scale Attentive Interaction Networks for Chinese Medical Question Answer Selection

Cited by 30 publications

References 19 publications

ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation

ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation

Recent Advances in Automated Question Answering In Biomedical Domain

Different Contextual Window Sizes Based RNNs for Multimodal Emotion Detection in Interactive Conversations

Contact Info

Product

Resources

About