Towards Automatic Generation of Shareable Synthetic Clinical Notes Using Neural Language Models

Melamud, Oren; Shivade, Chaitanya

doi:10.18653/v1/w19-1905

Cited by 19 publications

(28 citation statements)

References 34 publications

(41 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We compare two state-of-the-art language modeling approaches for the generation of synthetic EHR notes: a Long Short-Term Memory (LSTM) network [7] and a transformerbased network (GPT-2) [8]. To train these language models, we use a large and heterogeneous corpus of one million Dutch EHR notes.…”

Section: [<Namestart> Maria <Nameend>]mentioning

confidence: 99%

“…The generation of synthetic EHR text for use in medical NLP is still at an early stage [3]. Most studies focus on the creation of English EHR text, using hospital discharge summaries from the MIMIC-III database [7,8,13,14]. In addition, a corpus of English Mental Health Records was explored [15].…”

Section: Generating Synthetic Ehr Notesmentioning

confidence: 99%

“…Decoding from language models is the predominant approach in related work to generate synthetic text. Approaches include unigram-language models and LSTMs [7], as well as transformer-based methods such as GPT-2 [13][14][15]. In particular, Amin-Nejad et al [8] concluded that GPT-2 was suitable for text generation in a low-resource scenario.…”

Section: Generating Synthetic Ehr Notesmentioning

confidence: 99%

“…For example, Amin-Nejad et al [8] and Liu [13] used similarity metrics as intrinsic measure to compare real and synthetic notes, but did not draw further conclusions on privacy. Melamud and Shivade [7] propose an empirical measure to quantify the risk of information leakage based on differential privacy. However, the calculation of this measure requires training a prohibitively large amount of models and does not directly provide information on the privacy of the generated data itself.…”

Section: Evaluating Privacy Of Synthetic Ehr Notesmentioning

confidence: 99%

See 3 more Smart Citations

Generating Synthetic Training Data for Supervised De-Identification of Electronic Health Records

Libbi

Trienes

Trieschnigg³

et al. 2021

Future Internet

View full text Add to dashboard Cite

A major hurdle in the development of natural language processing (NLP) methods for Electronic Health Records (EHRs) is the lack of large, annotated datasets. Privacy concerns prevent the distribution of EHRs, and the annotation of data is known to be costly and cumbersome. Synthetic data presents a promising solution to the privacy concern, if synthetic data has comparable utility to real data and if it preserves the privacy of patients. However, the generation of synthetic text alone is not useful for NLP because of the lack of annotations. In this work, we propose the use of neural language models (LSTM and GPT-2) for generating artificial EHR text jointly with annotations for named-entity recognition. Our experiments show that artificial documents can be used to train a supervised named-entity recognition model for de-identification, which outperforms a state-of-the-art rule-based baseline. Moreover, we show that combining real data with synthetic data improves the recall of the method, without manual annotation effort. We conduct a user study to gain insights on the privacy of artificial text. We highlight privacy risks associated with language models to inform future research on privacy-preserving automated text generation and metrics for evaluating privacy-preservation during text generation.

show abstract

Section: [<Namestart> Maria <Nameend>]mentioning

confidence: 99%

Section: Generating Synthetic Ehr Notesmentioning

confidence: 99%

Section: Generating Synthetic Ehr Notesmentioning

confidence: 99%

Section: Evaluating Privacy Of Synthetic Ehr Notesmentioning

confidence: 99%

See 2 more Smart Citations

Generating Synthetic Training Data for Supervised De-Identification of Electronic Health Records

Libbi

Trienes

Trieschnigg³

et al. 2021

Future Internet

View full text Add to dashboard Cite

show abstract

“…Medical text generation has been a hot topic recently, such as electronic medical record (EMR) generation (Guan et al, 2018), medical question generation , clinical notes generation (Melamud and Shivade, 2019), etc. However, compared to the research in the general domain, there is still a lot of space for exploration, especially with the assistance of specific knowledge graph.…”

Section: Introductionmentioning

confidence: 99%

MedWriter: Knowledge-Aware Medical Text Generation

Pan¹,

Chen²,

Peng³

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

View full text Add to dashboard Cite

To exploit the domain knowledge to guarantee the correctness of generated text has been a hot topic in recent years, especially for high professional domains such as medical. However, most of recent works only consider the information of unstructured text rather than structured information of the knowledge graph. In this paper, we focus on the medical topic-to-text generation task and adapt a knowledge-aware text generation model to the medical domain, named MedWriter, which not only introduces the specific knowledge from the external MKG but also is capable of learning graph-level representation. We conduct experiments on a medical literature dataset collected from medical journals, each of which has a set of topic words, an abstract of medical literature and a corresponding knowledge graph from CMeKG. Experimental results demonstrate incorporating knowledge graph into generation model can improve the quality of the generated text and has robust superiority over the competitor methods.

show abstract

Synthetic Data for Deep Learning

Nikolenko

2021

Springer Optimization and Its Applications

185

View full text Add to dashboard Cite

Aims and ScopeOptimization has continued to expand in all directions at an astonishing rate. New algorithmic and theoretical techniques are continually developing and the diffusion into other disciplines is proceeding at a rapid pace, with a spot light on machine learning, artificial intelligence, and quantum computing. Our knowledge of all aspects of the field has grown even more profound. At the same time, one of the most striking trends in optimization is the constantly increasing emphasis on the interdisciplinary nature of the field. Optimization has been a basic tool in areas not limited to applied mathematics, engineering, medicine, economics, computer science, operations research, and other sciences.The series Springer Optimization and Its Applications (SOIA) aims to publish state-of-the-art expository works (monographs, contributed volumes, textbooks, handbooks) that focus on theory, methods, and applications of optimization. Topics covered include, but are not limited to, nonlinear optimization, combinatorial optimization, continuous optimization, stochastic optimization, Bayesian optimization, optimal control, discrete optimization, multi-objective optimization, and more. New to the series portfolio include Works at the intersection of optimization and machine learning, artificial intelligence, and quantum computing.Volumes from this series are indexed by Web of Science, zbMATH, Mathematical Reviews, and SCOPUS.

show abstract

Towards Automatic Generation of Shareable Synthetic Clinical Notes Using Neural Language Models

Cited by 19 publications

References 34 publications

Generating Synthetic Training Data for Supervised De-Identification of Electronic Health Records

Generating Synthetic Training Data for Supervised De-Identification of Electronic Health Records

MedWriter: Knowledge-Aware Medical Text Generation

Synthetic Data for Deep Learning

Contact Info

Product

Resources

About