Transformer-based Multi-target Regression on Electronic Health Records for Primordial Prevention of Cardiovascular Disease

Poulain, Raphael; Gupta, Mehak; Foraker, Randi E.; Beheshti, Rahmatollah

doi:10.1109/bibm52615.2021.9669441

Cited by 3 publications

(1 citation statement)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As of now, Transformer has emerged as the dominant method for encoding sequences, capable of mapping temporally-ordered tokens from different types of sequential data to latent representation space. Examples include textual data [5] and electronic health data [16]. Our sequential pattern encoder is built upon the Transformer, inspired by the effectiveness of this approach in modeling item sequence in [20,24,33].…”

Section: Sequential Pattern Encodingmentioning

confidence: 99%

Debiased Contrastive Learning for Sequential Recommendation

Yang¹,

Huang²,

Xia³

et al. 2023

Proceedings of the ACM Web Conference 2023

View full text Add to dashboard Cite

Current sequential recommender systems are proposed to tackle the dynamic user preference learning with various neural techniques, such as Transformer and Graph Neural Networks (GNNs). However, inference from the highly sparse user behavior data may hinder the representation ability of sequential pattern encoding. To address the label shortage issue, contrastive learning (CL) methods are proposed recently to perform data augmentation in two fashions: (i) randomly corrupting the sequence data (e.g., stochastic masking, reordering); (ii) aligning representations across pre-defined contrastive views. Although effective, we argue that current CL-based methods have limitations in addressing popularity bias and disentangling of user conformity and real interest. In this paper, we propose a new Debiased Contrastive learning paradigm for Recommendation (DCRec) that unifies sequential pattern encoding with global collaborative relation modeling through adaptive conformity-aware augmentation. This solution is designed to tackle the popularity bias issue in recommendation systems. Our debiased contrastive learning framework effectively captures both the patterns of item transitions within sequences and the dependencies between users across sequences. Our experiments on various real-world datasets have demonstrated that DCRec significantly outperforms state-ofthe-art baselines, indicating its efficacy for recommendation. To facilitate reproducibility of our results, we make our implementation of DCRec publicly available at: https://github.com/HKUDS/DCRec.

show abstract

Section: Sequential Pattern Encodingmentioning

confidence: 99%

Debiased Contrastive Learning for Sequential Recommendation

Yang¹,

Huang²,

Xia³

et al. 2023

Proceedings of the ACM Web Conference 2023

View full text Add to dashboard Cite

show abstract

Self-supervised representation learning for clinical decision making using EHR categorical data: a scoping review

ZHENG,

BENSAHLA,

BJELOGRLIC

et al. 2024

Preprint

View full text Add to dashboard Cite

The widespread adoption of Electronic Health Records (EHRs) and deep learning, particularly through Self-Supervised Representation Learning (SSRL) for categorical data, has transformed clinical decision-making. This scoping review, following PRISMA-ScR guidelines, examines 46 studies published from January 2019 to April 2024 across databases including PubMed, MEDLINE, Embase, ACM, and Web of Science, focusing on SSRL for unlabeled categorical EHR data. The review systematically assesses research trends in building efficient representations for medical tasks, identifying major trends in model families: Transformer-based (43%), Autoencoder-based (28%), and Graph Neural Network-based (17%) models. The analysis highlights scenarios where healthcare institutions can leverage or develop SSRL technologies. It also addresses current limitations in assessing the impact of these technologies and identifies research opportunities to enhance their influence on clinical practice.

show abstract

Enhancing Patient Outcome Prediction through Deep Learning with Sequential Diagnosis Codes from structural EHR: A systematic review (Preprint)

Hama,

Alsaleh,

Allery

et al. 2024

Preprint

View full text Add to dashboard Cite

BACKGROUND There has been a rapid growth in the application of structured Electronic Health Records (EHRs) to healthcare systems, where huge amounts of diagnosis codes presenting the temporal event of the patient are collected. In the era of artificial intelligence, many models, especially Deep Learning (DL), are applied for patient outcome prediction. This systematic review aimed to identify DL models developed for sequential diagnosis codes for patient outcome prediction. OBJECTIVE The main objective of this systematic review is to identify and summarise existing DL studies predicting patient outcome using sequences of diagnosis codes, as a key part of their predictors. Additionally, this study also investigates the challenge of generalisability and explainability of the predictive models. METHODS In this review, we identified all relevant studies by using the following four databases: PubMed, Embase, IEEE Xplore, and Web of Science. After that, we evaluated the included papers in various aspects: Deep learning techniques, characteristics of the dataset, prediction tasks, performance evaluation, generalizability, and explainability. We also assessed the risk of bias (PROBAST) and the concern of applicability. RESULTS In this review, 84 papers met the eligibility criteria and were selected, which showed the growing trend in this research area. Recurrent neural networks (RNN) (and their derivatives) (n = 47; 57.3%) and Transformers (n = 22; 26.8%) were the most popular architectures in DL-based models. Most studies present their input feature as sequence of visit embedding (n = 45; 53.6%). For the prediction tasks, the most common one is next visit diagnosis (n = 30; 23.4%), followed by heart failure (18; 14.1%), and mortality (n = 17; 13.3%). Only 7 studies evaluated their models in terms of generalisability. A positive correlation was observed between training sample size and model performance (AUROC) (p-value < 0.05). However, about 70% of included studies were found to have high risk of bias. CONCLUSIONS The application of deep learning in sequence of diagnosis has demonstrated remarkable promise in predicting patient outcomes. Using multiple types of features and integration of time intervals was found to improve the predictive performance. Addressing challenges related to generalisation and explainability will be instrumental in unlocking the full potential of DL for enhancing healthcare outcomes and patient care. CLINICALTRIAL This review was registered on PROSPERO (CRD42023434032).

show abstract

Transformer-based Multi-target Regression on Electronic Health Records for Primordial Prevention of Cardiovascular Disease

Cited by 3 publications

References 22 publications

Debiased Contrastive Learning for Sequential Recommendation

Debiased Contrastive Learning for Sequential Recommendation

Self-supervised representation learning for clinical decision making using EHR categorical data: a scoping review

Enhancing Patient Outcome Prediction through Deep Learning with Sequential Diagnosis Codes from structural EHR: A systematic review (Preprint)

Contact Info

Product

Resources

About