Clinical utility of automatic phenotype annotation in unstructured clinical notes: intensive care unit use

Zhang, Jingqing; Trujillo, Luis Daniel Bolanos; Tanwar, Ashwani; Ive, Julia; Gupta, Vibhor; Guo, Yike

doi:10.1136/bmjhci-2021-100519

Cited by 2 publications

(2 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unstructured data constitutes 80% of all EHR data (Kong, 2019;Mahbub et al, 2022) and can potentially contain information that is otherwise not present elsewhere in the patient's EHR (Zhang et al, 2022). They can, therefore be exploited by computational models to infer more information about a patient or develop predictive models for patient monitoring.…”

Section: Clinical Notes In Electronic Health Records (Ehr)mentioning

confidence: 99%

“…There are numerous examples in the literature where clinical notes have been used, sometimes in conjunction with structured data, to develop diagnostic or predictive models. Some examples include adverse drug effects (Dandala et al, 2019;Mahendran & McInnes, 2021), self-harm and drug abuse prediction (Obeid et al, 2020;Ridgway et al, 2021), hospitalisation and readmission risk (Huang et al, 2019;Song et al, 2022), mortality prediction Ye et al, 2020), and automatic phenotype annotation (Zhang et al, 2022).…”

Section: Clinical Notes In Electronic Health Records (Ehr)mentioning

confidence: 99%

See 1 more Smart Citation

Lightweight Transformers for Clinical Natural Language Processing

Rohanian¹,

Mohammadmahdi²,

Hannah³

et al. 2023

Preprint

View full text Add to dashboard Cite

Specialised pre-trained language models are becoming more frequent in NLP since they can potentially outperform models trained on generic texts. BioBERT (Sanh et al., 2019) and BioClinicalBERT (Alsentzer et al., 2019) are two examples of such models that have shown promise in medical NLP tasks. Many of these models are overparametrised and resource-intensive, but thanks to techniques like Knowledge Distillation (KD), it is possible to create smaller versions that perform almost as well as their larger counterparts. In this work, we specifically focus on development of compact language models for processing clinical texts (i.e. progress notes, discharge summaries etc). We developed a number of efficient lightweight clinical transformers using knowledge distillation and continual learning, with the number of parameters ranging from 15 million to 65 million. These models performed comparably to larger models such as BioBERT and ClinicalBioBERT and significantly outperformed other compact models trained on general or biomedical data. Our extensive evaluation was done across several standard datasets and covered a wide range of clinical text-mining tasks, including Natural Language Inference, Relation Extraction, Named Entity Recognition, and Sequence Classification. To our knowledge, this is the first comprehensive study specifically focused on creating efficient and compact transformers for clinical NLP tasks. The models and code used in this study can be found on our Huggingface profile at https: //huggingface.co/nlpie and Github page at https://github.com/ nlpie-research/Lightweight-Clinical-Transformers, respectively, promoting reproducibility of our results.

show abstract

Section: Clinical Notes In Electronic Health Records (Ehr)mentioning

confidence: 99%

Section: Clinical Notes In Electronic Health Records (Ehr)mentioning

confidence: 99%

Lightweight Transformers for Clinical Natural Language Processing

Rohanian¹,

Mohammadmahdi²,

Hannah³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

Lightweight transformers for clinical natural language processing

Rohanian,

Nouriborji,

Jauncey

et al. 2024

Nat. Lang. Eng.

View full text Add to dashboard Cite

Specialised pre-trained language models are becoming more frequent in Natural language Processing (NLP) since they can potentially outperform models trained on generic texts. BioBERT (Sanh et al., Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv: 1910.01108, 2019) and BioClinicalBERT (Alsentzer et al., Publicly available clinical bert embeddings. In Proceedings of the 2nd Clinical Natural Language Processing Workshop, pp. 72–78, 2019) are two examples of such models that have shown promise in medical NLP tasks. Many of these models are overparametrised and resource-intensive, but thanks to techniques like knowledge distillation, it is possible to create smaller versions that perform almost as well as their larger counterparts. In this work, we specifically focus on development of compact language models for processing clinical texts (i.e. progress notes, discharge summaries, etc). We developed a number of efficient lightweight clinical transformers using knowledge distillation and continual learning, with the number of parameters ranging from $15$ million to $65$ million. These models performed comparably to larger models such as BioBERT and ClinicalBioBERT and significantly outperformed other compact models trained on general or biomedical data. Our extensive evaluation was done across several standard datasets and covered a wide range of clinical text-mining tasks, including natural language inference, relation extraction, named entity recognition and sequence classification. To our knowledge, this is the first comprehensive study specifically focused on creating efficient and compact transformers for clinical NLP tasks. The models and code used in this study can be found on our Huggingface profile at https://huggingface.co/nlpie and Github page at https://github.com/nlpie-research/Lightweight-Clinical-Transformers, respectively, promoting reproducibility of our results.

show abstract

Clinical utility of automatic phenotype annotation in unstructured clinical notes: intensive care unit use

Cited by 2 publications

References 30 publications

Lightweight Transformers for Clinical Natural Language Processing

Lightweight Transformers for Clinical Natural Language Processing

Lightweight transformers for clinical natural language processing

Contact Info

Product

Resources

About