Development and validation of a prediction model for actionable aspects of frailty in the text of clinicians’ encounter notes

Martin, Jacob A.; Crane-Droesch, Andrew; Lapite, F.; Puhl, Joseph C; Kmiec, Tyler E; Silvestri, J.; Ungar, Lyle H.; Kinosian, Bruce; Himes, Blanca E.; Hubbard, Rebecca A.; Diamond, Joshua M.; Ahya, Vivek N.; Sims, Michael W.; Halpern, Scott D.; Weissman, Gary E.

doi:10.1093/jamia/ocab248

Cited by 5 publications

(13 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Overall, application papers are the most common type and primarily utilize supervised learning methods. Applications developed phenotyping algorithms for diverse uses, including development of clinical decision support tools [51,93,95] and phenome-wide association studies [66,68]. Among methods articles, approaches to alleviate the burden of labeling, such as semi-supervised and weakly-supervised learning, are the most prevalent.…”

Section: Resultsmentioning

confidence: 99%

“…feature engineering or labeling). Within the articles, corpora included notes from EHRs [33,83,[90][91][92] as well as external sources such as biomedical publications [34,72,[93][94][95][96] and articles from Wikipedia and Google News [32,69,97].…”

Section: Data Typesmentioning

confidence: 99%

“…Among the methods used to train word embeddings, Word2vec [98] and Bidirectional Encoder Representations from Transformers (BERT) and variants [99][100][101][102] are the most frequently used (Supplementary Material Table S9). Word embeddings typically serve as the input layer to phenotyping algorithms using deep learning models and have also been used to account for ambiguous abbreviations and spelling errors in clinical notes [25,27,29,32,34,42,69,72,[74][75][76][77]83,[93][94][95][96][97][103][104][105][106][107][108][109][110][111][112][113][114].…”

Section: Data Typesmentioning

confidence: 99%

“…33 of the 72 supervised learning articles employ deep learning, with 31 of these articles using word embeddings as the input layer. Feed-forward neural networks are the least frequently used architecture [25,32,108,111,118,133], while recurrent neural networks (RNNS), [27,34,59,69,[74][75][76]85,90,94,95,103,[105][106][107]109,110,123,126,130,134], convolutional neural networks (CNNs) [33,75,83,91,93,94,103,106,107,112,126,[134][135][136], and their variants are frequently used (Supplementary Material Table S12). RNNs are particularly well suited for sequential data from longitudinal EHRs.…”

Section: Deep Supervised Learningmentioning

confidence: 99%

“…In spite of the advantages of deep learning, traditional supervised learning methods have outperformed their deep learning counterparts. For example, Martin et al found that a penalized elastic net logistic regression consistently outperformed FFNNs for identifying frailty aspects such as musculoskeletal problems and respiratory impairment (F1-score Elastic net 0.51-0.65, F1-score FFNN 0.24-0.62) [95]. Several authors also commented on challenges related to model interpretability and transportability of deep learning models across sites [25,85,91,130].…”

Section: Deep Supervised Learningmentioning

confidence: 99%

See 4 more Smart Citations

Machine Learning Approaches for Electronic Health Records Phenotyping: A Methodical Review

Yang

Varghese²,

Stephenson

et al. 2022

Preprint

View full text Add to dashboard Cite

ObjectiveAccurate and rapid methods for phenotyping are a prerequisite to realizing the potential of electronic health records (EHRs) data for clinical and translational research. This study reviews the literature on machine learning (ML) approaches for phenotyping with respect to the phenotypes considered, the data sources and methods used, and the contributions within the wider context of EHR-based research.Materials and MethodsWe searched for relevant articles in PubMed and Web of Science published between January 1, 2018 and April 14, 2022. After screening, we collected data on 52 variables across 106 selected articles.ResultsML-based methods were developed for 156 unique phenotypes, primarily using EHR data from a single institution or health system. 72 of 106 articles leveraged unstructured data in clinical notes. In terms of methodology, supervised learning is the most prevalent ML paradigm (n = 64, 60.4%), with half of the articles employing deep learning. Semi-supervised and weakly-supervised approaches were applied to reduce the burden of obtaining gold-standard labeled data (n = 21, 19.8%), while unsupervised learning was used for phenotype discovery (n = 20, 18.9%). Federated learning has been applied to develop algorithms across multiple institutions while preserving data privacy (n = 2, 1.9%).DiscussionWhile the use of ML for phenotyping is growing, most articles applied traditional supervised ML to characterize the presence of common, chronic conditions.ConclusionContinued research in ML-based methods is warranted, with particular attention to the development of advanced methods for complex phenotypes and standards for reporting and evaluating phenotyping algorithms.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Data Typesmentioning

confidence: 99%

Section: Data Typesmentioning

confidence: 99%

Section: Deep Supervised Learningmentioning

confidence: 99%

Section: Deep Supervised Learningmentioning

confidence: 99%

See 3 more Smart Citations

Machine Learning Approaches for Electronic Health Records Phenotyping: A Methodical Review

Yang

Varghese²,

Stephenson

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

Machine learning approaches for electronic health records phenotyping: a methodical review

Yang

Varghese

Stephenson

et al. 2022

Journal of the American Medical Informatics Association

View full text Add to dashboard Cite

Objective Accurate and rapid phenotyping is a prerequisite to leveraging electronic health records for biomedical research. While early phenotyping relied on rule-based algorithms curated by experts, machine learning (ML) approaches have emerged as an alternative to improve scalability across phenotypes and healthcare settings. This study evaluates ML-based phenotyping with respect to (1) the data sources used, (2) the phenotypes considered, (3) the methods applied, and (4) the reporting and evaluation methods used. Materials and methods We searched PubMed and Web of Science for articles published between 2018 and 2022. After screening 850 articles, we recorded 37 variables on 100 studies. Results Most studies utilized data from a single institution and included information in clinical notes. Although chronic conditions were most commonly considered, ML also enabled the characterization of nuanced phenotypes such as social determinants of health. Supervised deep learning was the most popular ML paradigm, while semi-supervised and weakly supervised learning were applied to expedite algorithm development and unsupervised learning to facilitate phenotype discovery. ML approaches did not uniformly outperform rule-based algorithms, but deep learning offered a marginal improvement over traditional ML for many conditions. Discussion Despite the progress in ML-based phenotyping, most articles focused on binary phenotypes and few articles evaluated external validity or used multi-institution data. Study settings were infrequently reported and analytic code was rarely released. Conclusion Continued research in ML-based phenotyping is warranted, with emphasis on characterizing nuanced phenotypes, establishing reporting and evaluation standards, and developing methods to accommodate misclassified phenotypes due to algorithm errors in downstream applications.

show abstract

Development of a Machine Learning Model to Identify Colorectal Cancer Stage in Medicare Claims

Finn

Sharpe

Tong

et al. 2023

JCO Clinical Cancer Informatics

View full text Add to dashboard Cite

PURPOSE Staging information is essential for colorectal cancer research. Medicare claims are an important source of population-level data but currently lack oncologic stage. We aimed to develop a claims-based model to identify stage at diagnosis in patients with colorectal cancer. METHODS We included patients age 66 years or older with colorectal cancer in the SEER-Medicare registry. Using patients diagnosed from 2014 to 2016, we developed models (multinomial logistic regression, elastic net regression, and random forest) to classify patients into stage I-II, III, or IV on the basis of demographics, diagnoses, and treatment utilization identified in Medicare claims. Models developed in a training cohort (2014-2016) were applied to a testing cohort (2017), and performance was evaluated using cancer stage listed in the SEER registry as the reference standard. RESULTS The cohort of patients with 30,543 colorectal cancer included 14,935 (48.9%) patients with stage I-II, 9,203 (30.1%) with stage III, and 6,405 (21%) with stage IV disease. A claims-based model using elastic net regression had a scaled Brier score (SBS) of 0.45 (95% CI, 0.43 to 0.46). Performance was strongest for classifying stage IV (SBS, 0.62; 95% CI, 0.59 to 0.64; sensitivity, 93%; 95% CI, 91 to 94) followed by stage I-II (SBS, 0.45; 95% CI, 0.44 to 0.47; sensitivity, 86%; 95% CI, 85 to 76) and stage III (SBS, 0.32; 95% CI, 0.30 to 0.33; sensitivity, 62%; 95% CI, 61 to 64). CONCLUSION Machine learning models effectively classified colorectal cancer stage using Medicare claims. These models extend the ability of claims-based research to risk-adjust and stratify by stage.

show abstract

Development and validation of a prediction model for actionable aspects of frailty in the text of clinicians’ encounter notes

Cited by 5 publications

References 71 publications

Machine Learning Approaches for Electronic Health Records Phenotyping: A Methodical Review

Machine Learning Approaches for Electronic Health Records Phenotyping: A Methodical Review

Machine learning approaches for electronic health records phenotyping: a methodical review

Development of a Machine Learning Model to Identify Colorectal Cancer Stage in Medicare Claims

Contact Info

Product

Resources

About