A Transformer-Based Model Trained on Large Scale Claims Data for Prediction of Severe COVID-19 Disease Progression

Lentzen, Manuel; Lindén, Thomas; Veeranki, Sai; Madan, Sumit; Kramer, Diether; Leodolter, Werner; Fröhlich, Holger

doi:10.1109/jbhi.2023.3288768

Cited by 8 publications

(2 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…age, sex, education) or molecular data modalities commonly available in biobanks (e.g. genomics, proteomics) 37 .…”

Section: Discussionmentioning

confidence: 99%

Deep representation learning for clustering longitudinal survival data from electronic health records

Qiu,

Hu,

et al. 2024

Preprint

View full text Add to dashboard Cite

Precision medicine can be defined as providing the right treatment to the right patient at the right time, and it requires the ability to identify clinically relevant patient subgroups with high accuracy. The increasing availability of large-scale electronic health records (EHR) datasets has provided major opportunities for artificial intelligence and machine learning in mining such complex datasets for identifying novel disease subtypes. However, disease subtypes often exist in the context of certain disease-relevant risk events, and current efforts have been limited by analyzing clustering and event risk independently, resulting in subgroups that still display great heterogeneity in event risk and/or underlying molecular mechanisms. To address this problem, we developed TransVarSur (Transformer Variational Survival modeling). TransVarSur integrates a Transformer-based Gaussian mixture variational autoencoder with time-to-event modeling to capture complex relationships between cluster-specific EHR trajectories and survival times. We validated TransVarSur by showing superior performance relative to baseline methods, on both synthetic and real-world benchmark datasets with known ground-truth clustering. We then applied TransVarSur to 1908 Crohn's disease patients from the UK Biobank and successfully identified four clusters displaying both divergent EHR trajectories and divergent progression towards the risk event intestinal obstruction. A further analysis of the clusters revealed known clinical and genetic factors relevant in Crohn's disease and progression to intestinal obstruction. In conclusion, we demonstrated TransVarSur's ability to accurately stratify a patient population into clinically relevant, risk-associated subgroups. Hence, it can be a powerful tool in the development of precision medicine approaches.

show abstract

“…age, sex, education) or molecular data modalities commonly available in biobanks (e.g. genomics, proteomics) 37 .…”

Section: Discussionmentioning

confidence: 99%

Deep representation learning for clustering longitudinal survival data from electronic health records

Qiu,

Hu,

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Specifically, transformers learn context by tracking relationships in sequential data such as words in a sentence. Typically, transformer-based models are trained in 2 phases: the pretraining phase focuses on generic representation learning, and the transfer learning phase focuses on adjusting the model to an application-specific prediction task [ 4 ]. The pretrained models, which are often trained on large data sets (eg, Wikipedia, Reddit, biomedical literature, or public medical data sets), are tuned to be used for a wider set of tasks and can be fine-tuned for specific tasks [ 5 ].…”

Section: Introductionmentioning

confidence: 99%

Potential of Large Language Models in Health Care: Delphi Study

Denecke,

May,

Rivera Romero

2024

J Med Internet Res

View full text Add to dashboard Cite

Background A large language model (LLM) is a machine learning model inferred from text data that captures subtle patterns of language use in context. Modern LLMs are based on neural network architectures that incorporate transformer methods. They allow the model to relate words together through attention to multiple words in a text sequence. LLMs have been shown to be highly effective for a range of tasks in natural language processing (NLP), including classification and information extraction tasks and generative applications. Objective The aim of this adapted Delphi study was to collect researchers’ opinions on how LLMs might influence health care and on the strengths, weaknesses, opportunities, and threats of LLM use in health care. Methods We invited researchers in the fields of health informatics, nursing informatics, and medical NLP to share their opinions on LLM use in health care. We started the first round with open questions based on our strengths, weaknesses, opportunities, and threats framework. In the second and third round, the participants scored these items. Results The first, second, and third rounds had 28, 23, and 21 participants, respectively. Almost all participants (26/28, 93% in round 1 and 20/21, 95% in round 3) were affiliated with academic institutions. Agreement was reached on 103 items related to use cases, benefits, risks, reliability, adoption aspects, and the future of LLMs in health care. Participants offered several use cases, including supporting clinical tasks, documentation tasks, and medical research and education, and agreed that LLM-based systems will act as health assistants for patient education. The agreed-upon benefits included increased efficiency in data handling and extraction, improved automation of processes, improved quality of health care services and overall health outcomes, provision of personalized care, accelerated diagnosis and treatment processes, and improved interaction between patients and health care professionals. In total, 5 risks to health care in general were identified: cybersecurity breaches, the potential for patient misinformation, ethical concerns, the likelihood of biased decision-making, and the risk associated with inaccurate communication. Overconfidence in LLM-based systems was recognized as a risk to the medical profession. The 6 agreed-upon privacy risks included the use of unregulated cloud services that compromise data security, exposure of sensitive patient data, breaches of confidentiality, fraudulent use of information, vulnerabilities in data storage and communication, and inappropriate access or use of patient data. Conclusions Future research related to LLMs should not only focus on testing their possibilities for NLP-related tasks but also consider the workflows the models could contribute to and the requirements regarding quality, integration, and regulations needed for successful implementation in practice.

show abstract

Towards revolutionizing precision healthcare: A systematic literature review of artificial intelligence methods in precision medicine

Abbaoui,

Retal,

El Bhiri

et al. 2024

Informatics in Medicine Unlocked

View full text Add to dashboard Cite

A Transformer-Based Model Trained on Large Scale Claims Data for Prediction of Severe COVID-19 Disease Progression

Cited by 8 publications

References 50 publications

Deep representation learning for clustering longitudinal survival data from electronic health records

Deep representation learning for clustering longitudinal survival data from electronic health records

Potential of Large Language Models in Health Care: Delphi Study

Towards revolutionizing precision healthcare: A systematic literature review of artificial intelligence methods in precision medicine

Contact Info

Product

Resources

About