SECNLP: A survey of embeddings in clinical natural language processing

Kalyan, Katikapalli Subramanyam; Sangeetha, S.

doi:10.1016/j.jbi.2019.103323

Cited by 63 publications

(56 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, although DL is celebrated for not requiring manual feature engineering, the effects of proper hyperparameter tuning on DNNs [93] remain an issue for DL [94]. Apart from these intrinsic problems, Kalyan and Sangeetha [95] and Khattak et al [96] refer to extrinsic drawbacks of neural networks, such as opaque encodings (resulting in lacking interpretability) or limited transferability of large models (hindering knowledge distillation for smaller models).…”

Section: Discussionmentioning

confidence: 99%

Medical Information Extraction in the Age of Deep Learning

Hahn

Oleynik

2020

Yearb Med Inform

View full text Add to dashboard Cite

Objectives: We survey recent developments in medical Information Extraction (IE) as reported in the literature from the past three years. Our focus is on the fundamental methodological paradigm shift from standard Machine Learning (ML) techniques to Deep Neural Networks (DNNs). We describe applications of this new paradigm concentrating on two basic IE tasks, named entity recognition and relation extraction, for two selected semantic classes—diseases and drugs (or medications)—and relations between them. Methods: For the time period from 2017 to early 2020, we searched for relevant publications from three major scientific communities: medicine and medical informatics, natural language processing, as well as neural networks and artificial intelligence. Results: In the past decade, the field of Natural Language Processing (NLP) has undergone a profound methodological shift from symbolic to distributed representations based on the paradigm of Deep Learning (DL). Meanwhile, this trend is, although with some delay, also reflected in the medical NLP community. In the reporting period, overwhelming experimental evidence has been gathered, as illustrated in this survey for medical IE, that DL-based approaches outperform non-DL ones by often large margins. Still, small-sized and access-limited corpora create intrinsic problems for data-greedy DL as do special linguistic phenomena of medical sublanguages that have to be overcome by adaptive learning strategies. Conclusions: The paradigm shift from (feature-engineered) ML to DNNs changes the fundamental methodological rules of the game for medical NLP. This change is by no means restricted to medical IE but should also deeply influence other areas of medical informatics, either NLP- or non-NLP-based.

show abstract

Section: Discussionmentioning

confidence: 99%

Medical Information Extraction in the Age of Deep Learning

Hahn

Oleynik

2020

Yearb Med Inform

View full text Add to dashboard Cite

show abstract

“…As many studies in the field of clinical NER have shown [21] , [89] , [90] , combining different types of embeddings together or using the most advanced ones, with different techniques [54] , [91] , can be a useful method to take advantage of their different characteristics and achieve better performance. In detail, the concatenation technique was used, getting the stacked embedding

of each word as:

where

and

are respectively the MultiBPEmb word embedding and a type of Flair contextual string embedding and

is a weight matrix to remap the original stacked embedding, hereinafter Original-Embedding , into a new trainable embedding, called Reprojected-Embedding .…”

Section: Methodsmentioning

confidence: 99%

Crosslingual named entity recognition for clinical de-identification applied to a COVID-19 Italian data set

Catelli

Gargiulo

Casola

et al. 2020

Applied Soft Computing

View full text Add to dashboard Cite

The COrona VIrus Disease 19 (COVID-19) pandemic required the work of all global experts to tackle it. Despite the abundance of new studies, privacy laws prevent their dissemination for medical investigations: through clinical de-identification, the Protected Health Information (PHI) contained therein can be anonymized so that medical records can be shared and published. The automation of clinical de-identification through deep learning techniques has proven to be less effective for languages other than English due to the scarcity of data sets. Hence a new Italian de-identification data set has been created from the COVID-19 clinical records made available by the Italian Society of Radiology (SIRM). Therefore, two multi-lingual deep learning systems have been developed for this low-resource language scenario: the objective is to investigate their ability to transfer knowledge between different languages while maintaining the necessary features to correctly perform the Named Entity Recognition task for de-identification. The systems were trained using four different strategies, using both the English Informatics for Integrating Biology & the Bedside (i2b2) 2014 and the new Italian SIRM COVID-19 data sets, then evaluated on the latter. These approaches have demonstrated the effectiveness of cross-lingual transfer learning to de-identify medical records written in a low resource language such as Italian, using one with high resources such as English.

show abstract

“…The misuse use of race, sex, age and other features is criticized in biomedical research due to the inappropriate use of these labels, unequal representation of different social groups and the neglect of other key identity factors [61][62][63]. Despite this limitation, we have used these terms below as they reflect the current data collected by the UK & USA census bureaus and continue to pervade the medical field.…”

Section: Demographic Labelsmentioning

confidence: 99%

Artificial Intelligence in mental health and the biases of language based models

Straw

Callison-Burch

2020

PLoS ONE

View full text Add to dashboard Cite

Background The rapid integration of Artificial Intelligence (AI) into the healthcare field has occurred with little communication between computer scientists and doctors. The impact of AI on health outcomes and inequalities calls for health professionals and data scientists to make a collaborative effort to ensure historic health disparities are not encoded into the future. We present a study that evaluates bias in existing Natural Language Processing (NLP) models used in psychiatry and discuss how these biases may widen health inequalities. Our approach systematically evaluates each stage of model development to explore how biases arise from a clinical, data science and linguistic perspective. Design/Methods A literature review of the uses of NLP in mental health was carried out across multiple disciplinary databases with defined Mesh terms and keywords. Our primary analysis evaluated biases within ‘GloVe’ and ‘Word2Vec’ word embeddings. Euclidean distances were measured to assess relationships between psychiatric terms and demographic labels, and vector similarity functions were used to solve analogy questions relating to mental health. Results Our primary analysis of mental health terminology in GloVe and Word2Vec embeddings demonstrated significant biases with respect to religion, race, gender, nationality, sexuality and age. Our literature review returned 52 papers, of which none addressed all the areas of possible bias that we identify in model development. In addition, only one article existed on more than one research database, demonstrating the isolation of research within disciplinary silos and inhibiting cross-disciplinary collaboration or communication. Conclusion Our findings are relevant to professionals who wish to minimize the health inequalities that may arise as a result of AI and data-driven algorithms. We offer primary research identifying biases within these technologies and provide recommendations for avoiding these harms in the future.

show abstract

SECNLP: A survey of embeddings in clinical natural language processing

Cited by 63 publications

References 55 publications

Medical Information Extraction in the Age of Deep Learning

Medical Information Extraction in the Age of Deep Learning

Crosslingual named entity recognition for clinical de-identification applied to a COVID-19 Italian data set

Artificial Intelligence in mental health and the biases of language based models

Contact Info

Product

Resources

About