Language models are an effective representation learning technique for electronic health record data

Steinberg, Ethan; Jung, Kenneth; Fries, Jason; Corbin, Conor K.; Pfohl, Stephen R.; Shah, Nigam H.

doi:10.1016/j.jbi.2020.103637

Cited by 66 publications

(65 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among the 25 studies applying feature-representation transfer, support vector machines and 'vanilla' neural networks were the most common methods used as the final model producing outputs based on the feature representations. Seven studies applied and compared multiple methods for this purpose [18][19][20][21][22][23][24].…”

Section: Resultsmentioning

confidence: 99%

Transfer learning for non-image data in clinical research: A scoping review

et al. 2022

View full text Add to dashboard Cite

Background Transfer learning is a form of machine learning where a pre-trained model trained on a specific task is reused as a starting point and tailored to another task in a different dataset. While transfer learning has garnered considerable attention in medical image analysis, its use for clinical non-image data is not well studied. Therefore, the objective of this scoping review was to explore the use of transfer learning for non-image data in the clinical literature. Methods and findings We systematically searched medical databases (PubMed, EMBASE, CINAHL) for peer-reviewed clinical studies that used transfer learning on human non-image data. We included 83 studies in the review. More than half of the studies (63%) were published within 12 months of the search. Transfer learning was most often applied to time series data (61%), followed by tabular data (18%), audio (12%) and text (8%). Thirty-three (40%) studies applied an image-based model to non-image data after transforming data into images (e.g. spectrograms). Twenty-nine (35%) studies did not have any authors with a health-related affiliation. Many studies used publicly available datasets (66%) and models (49%), but fewer shared their code (27%). Conclusions In this scoping review, we have described current trends in the use of transfer learning for non-image data in the clinical literature. We found that the use of transfer learning has grown rapidly within the last few years. We have identified studies and demonstrated the potential of transfer learning in clinical research in a wide range of medical specialties. More interdisciplinary collaborations and the wider adaption of reproducible research principles are needed to increase the impact of transfer learning in clinical research.

show abstract

Section: Resultsmentioning

confidence: 99%

Transfer learning for non-image data in clinical research: A scoping review

et al. 2022

View full text Add to dashboard Cite

show abstract

Section: Resultsmentioning

confidence: 99%

“…Tabular data is probably the most used data type within clinical research. However, we only identified 15 studies using transfer learning on tabular data covering very different fields in medicine: two-thirds of them were from genetics [98][99][100][101][102], pathology [103][104][105], and intensive care [18,106], while the remaining five were from surgery [17], neonatology [107], infectious disease [108], pulmonology [109], and pharmacology [110]. Oncological applications like classification of cancer and prediction of cancer survival were common among the studies in genetics or pathology.…”

Section: Tabular Datamentioning

confidence: 99%

Transfer learning for non-image data in clinical research: a scoping review

Ebbehøj

Thunbo

Andersen

et al. 2021

Preprint

View full text Add to dashboard Cite

Background Transfer learning is a form of machine learning where a pre-trained model trained on a specific task is reused as a starting point and tailored to another task in a different dataset. While transfer learning has garnered considerable attention in medical image analysis, its use for clinical non-image data is not well studied. Therefore, the objective of this scoping review was to explore the use of transfer learning for non-image data in the clinical literature. Methods and Findings We systematically searched medical databases (PubMed, EMBASE, CINAHL) for peer-reviewed clinical studies that used transfer learning on human non-image data. We included 83 studies in the review. More than half of the studies (63%) were published within 12 months of the search. Transfer learning was most often applied to time series data (61%), followed by tabular data (18%), audio (12%) and text (8%). Thirty-three (40%) studies applied an image-based model to non-image data after transforming data into images (e.g. spectrograms). Twenty-nine (35%) studies did not have any authors with a health-related affiliation. Many studies used publicly available datasets (66%) and models (49%), but fewer shared their code (27%). Conclusions In this scoping review, we have described current trends in the use of transfer learning for non-image data in the clinical literature. We found that the use of transfer learning has grown rapidly within the last few years. We have identified studies and demonstrated the potential of transfer learning in clinical research in a wide range of medical specialties. More interdisciplinary collaborations and the wider adaption of reproducible research principles are needed to increase the impact of transfer learning in clinical research.

show abstract

“…In 2018, Zhang et al used the Word2Vec approach in Patient2Vec [3] to learn a personalized representation for each patient to use it in future hospitalization prediction. In a recent research, Steinberg et al [4] suggest using word embeddings to represent medical codes and train their model to predict the medical codes contained in a medical visit using the codes contained in previous visits.…”

Section: Natural Language Processing and Patient Representation Learningmentioning

confidence: 99%

“…End-to-end models, like Patient2Vec [3], are trained on the specific task of predicting a defined medical outcome. Non end-to-end models, such as CLMBR [4], are models that first learn patient representations (in this case, by constructing a clinical language model) independently of any prediction task. The learned representations are then tested by using them as inputs in several clinical prediction models such as logistic regressions or gradient boosted trees.…”

Section: Learning Patient Representationsmentioning

confidence: 99%

Patient representation learning from EHR data

Grandpré

2021

Proceedings of the Canadian Conference on Artificial Intelligence

View full text Add to dashboard Cite

Language models are an effective representation learning technique for electronic health record data

Cited by 66 publications

References 19 publications

Transfer learning for non-image data in clinical research: A scoping review

Transfer learning for non-image data in clinical research: A scoping review

Transfer learning for non-image data in clinical research: a scoping review

Patient representation learning from EHR data

Contact Info

Product

Resources

About