Towards Medical Machine Reading Comprehension with Structural Knowledge and Plain Text

Li, Dongfang; Hu, Baotian; Chen, Qingcai; Peng, Weihua; Wang, Anqi

doi:10.18653/v1/2020.emnlp-main.111

Cited by 31 publications

(20 citation statements)

References 40 publications

(32 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ALBERT has been applied to some tasks, such as natural language inference [ 32 ], sentiment analysis [ 33 ], causality analysis [ 34 ], and medical machine reading [ 35 ]. The self-attention structure is the core part of the transformer mechanism.…”

Section: Methodsmentioning

confidence: 99%

ALBERT-Based Self-Ensemble Model With Semisupervised Learning and Data Augmentation for Clinical Semantic Textual Similarity Calculation: Algorithm Validation Study

Zhang

Zhou

2021

JMIR Med Inform

View full text Add to dashboard Cite

Background In recent years, with increases in the amount of information available and the importance of information screening, increased attention has been paid to the calculation of textual semantic similarity. In the field of medicine, electronic medical records and medical research documents have become important data resources for clinical research. Medical textual semantic similarity calculation has become an urgent problem to be solved. Objective This research aims to solve 2 problems—(1) when the size of medical data sets is small, leading to insufficient learning with understanding of the models and (2) when information is lost in the process of long-distance propagation, causing the models to be unable to grasp key information. Methods This paper combines a text data augmentation method and a self-ensemble ALBERT model under semisupervised learning to perform clinical textual semantic similarity calculations. Results Compared with the methods in the 2019 National Natural Language Processing Clinical Challenges Open Health Natural Language Processing shared task Track on Clinical Semantic Textual Similarity, our method surpasses the best result by 2 percentage points and achieves a Pearson correlation coefficient of 0.92. Conclusions When the size of medical data set is small, data augmentation can increase the size of the data set and improved semisupervised learning can boost the learning efficiency of the model. Additionally, self-ensemble methods improve the model performance. Our method had excellent performance and has great potential to improve related medical problems.

show abstract

Section: Methodsmentioning

confidence: 99%

ALBERT-Based Self-Ensemble Model With Semisupervised Learning and Data Augmentation for Clinical Semantic Textual Similarity Calculation: Algorithm Validation Study

Zhang

Zhou

2021

JMIR Med Inform

View full text Add to dashboard Cite

show abstract

“…However, it contains a relatively small variety of medical questions, automatically generated from clinical case reports. Recently, (Li et al, 2020) propose a multi-choice Chinese medical QA dataset, retrieving text snippets as the passage and the task only chooses an existing correct option from candidate set. Our work specifically focuses on the fine-grained medical MRC tasks and deep domain knowledge reasoning, with a manually constructed high-quality dataset released.…”

Section: Related Workmentioning

confidence: 99%

“…We evaluate CMedBERT on CMedMRC, and compare it against six strong baselines: DrQA (Chen et al, 2017), BERT base , ERNIE , KT-NET , MC-BERT (Zhang et al, 2020) and KMQA (Li et al, 2020). KT-NET is the first model to leverage rich knowledge to enhance pre-trained language models for MRC.…”

Section: Experimental Setupsmentioning

confidence: 99%

Knowledge-Empowered Representation Learning for Chinese Medical Reading Comprehension: Task, Model and Resources

Zhang¹,

Wang²,

Qiu³

et al. 2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

View full text Add to dashboard Cite

Machine Reading Comprehension (MRC) aims to extract answers to questions given a passage, which has been widely studied recently especially in open domains. However, few efforts have been made on closeddomain MRC, mainly due to the lack of largescale training data. In this paper, we introduce a multi-target MRC task for the medical domain, whose goal is to predict answers to medical questions and the corresponding support sentences from medical information sources simultaneously, in order to ensure the high reliability of medical knowledge serving. A high-quality dataset (more than 18k samples) is manually constructed for the purpose, named Multi-task Chinese Medical MRC dataset (CMedMRC), with detailed analysis conducted. We further propose a Chinese medical BERT model for the task (CMedBERT), which fuses medical knowledge into pre-trained language models by the dynamic fusion mechanism of heterogeneous features and the multi-task learning strategy. Experiments show that CMedBERT consistently outperforms strong baselines by fusing context-aware and knowledge-aware token representations. 1

show abstract

“…ALBERT has been applied to some tasks, such as natural language inference [32], sentiment analysis [33], causality analysis [34], and medical machine reading [35]. The self-attention structure is the core part of the transformer mechanism.…”

Section: Self-ensemble Albert Modelmentioning

confidence: 99%

ALBERT-Based Self-Ensemble Model With Semisupervised Learning and Data Augmentation for Clinical Semantic Textual Similarity Calculation: Algorithm Validation Study (Preprint)

Li¹,

Zhang²,

Zhou³

2020

Preprint

View full text Add to dashboard Cite

BACKGROUND In recent years, with the increase in the amount of information and the importance of information screening, increasing attention has been paid to the calculation of textual semantic similarity. In the medical field, with the rapid increase in electronic medical data, electronic medical records and medical research documents have become important data resources for medical clinical research. Medical textual semantic similarity calculation has become an urgent problem to be solved. The 2019 N2C2/OHNLP shared task Track on Clinical Semantic Textual Similarity is one of significant tasks for medical textual semantic similarity calculation. OBJECTIVE This research aims to solve two problems: 1) The size of medical datasets is small, which leads to the problem of insufficient learning with understanding of the models; 2) The data information will be lost in the process of long-distance propagation, which causes the models to be unable to grasp key information. METHODS This paper introduces a text data augmentation method and self-integrated ALBERT model under unsupervised learning to perform clinical textual semantic similarity calculation. RESULTS Compared with the competition methods the 2019 N2C2/OHNLP Track 1 ClinicalSTS , our method achieves state-of-the-art result with a value 0.92 of the Pearson correlation coefficient and surpasses the best result by 2%. CONCLUSIONS When the size of medical dataset is small, data augmentation and improved semi-supervised learning can increase the size of dataset and boost the learning efficiency of the model. Additionally, self-ensemble improves the model performance significantly.

show abstract

Towards Medical Machine Reading Comprehension with Structural Knowledge and Plain Text

Cited by 31 publications

References 40 publications

ALBERT-Based Self-Ensemble Model With Semisupervised Learning and Data Augmentation for Clinical Semantic Textual Similarity Calculation: Algorithm Validation Study

ALBERT-Based Self-Ensemble Model With Semisupervised Learning and Data Augmentation for Clinical Semantic Textual Similarity Calculation: Algorithm Validation Study

Knowledge-Empowered Representation Learning for Chinese Medical Reading Comprehension: Task, Model and Resources

ALBERT-Based Self-Ensemble Model With Semisupervised Learning and Data Augmentation for Clinical Semantic Textual Similarity Calculation: Algorithm Validation Study (Preprint)

Contact Info

Product

Resources

About