Unsupervised machine learning for the discovery of latent disease clusters and patient subgroups using electronic health records

Abstract

Machine learning has become ubiquitous and a key technology on mining electronic health records (EHRs) for facilitating clinical research and practice. Unsupervised machine learning, as opposed to supervised learning, has shown promise in identifying novel patterns and relations from EHRs without using human created labels. In this paper, we investigate the application of unsupervised machine learning models in discovering latent disease clusters and patient subgroups based on EHRs. We utilized Latent Dirichlet Allocation (LDA), a generative probabilistic model, and proposed a novel model named Poisson Dirichlet Model (PDM), which extends the LDA approach using a Poisson distribution to model patients' disease diagnoses and to alleviate age and sex factors by considering both observed and expected observations. In the empirical experiments, we evaluated LDA and PDM on three patient cohorts, namely Osteoporosis, Delirium/Dementia, and Chronic Obstructive Pulmonary Disease (COPD)/Bronchiectasis Cohorts, with EHR data retrieved from the Rochester Epidemiology Project (REP) medical records linkage system, for the discovery of latent disease clusters and patient subgroups. We compared the effectiveness of LDA and PDM in identifying latent disease clusters through the visualization of disease representations learned by two approaches. We also tested the performance of LDA and PDM in differentiating patient subgroups through survival analysis, as well as statistical analysis of demographics and Elixhauser Comorbidity Index (ECI) scores in those subgroups. The experimental results show that the proposed PDM could effectively identify distinguished disease clusters based on the latent patterns hidden in the EHR data by alleviating the impact of age and sex, and that LDA could stratify patients into more differentiable subgroups than PDM in terms of p-values. However, the subgroups discovered by PDM might imply the underlying patterns of diseases of greater interest in epidemiology research due to the alleviation of age and sex. Both unsupervised machine learning approaches could be leveraged to discover patient subgroups using EHRs but with different foci.

Machine Learning Solutions for Osteoporosis—A Review

“…( 98–111 ) Two of them used unsupervised learning to identify fracture and comorbidity risk groups, respectively. ( 98,99 ) Kruse and colleagues developed a fracture risk clustering model to categorize subgroups of patients at risk. ( 98 ) Wang and colleagues investigated osteoporotic patients' subgroups and their related comorbidity risk.…”

Machine Learning Applied to Clinical Laboratory Data Predicts Patient-Specific, Near-Term Relapse in Patients in Medication for Opioid Use Disorder Treatment

“…This data could be amenable to machine learning and data science methods now being applied in other areas of medicine (7; 8; 9; 10) , being tempered with the knowledge of how the underlying testing works, and the insights into patient behavior provided by expert clinicians.…”

Lab indicators standardization method for the regional healthcare platform: a case study on heart failure

“…The second step is to pick out lab indicators related to special disease and construct the special disease lab indicator KB. The last step is to named entity recognize [ 32 , 33 ] using the special disease indicator KB and turn text descriptions into structured data and map synonymous names into standard names.…”

“…The last step is the post-structure step. Each synonymous indicator is recognized using the heart failure indicator KB constructed above [ 32 ], and it is mapped to the standard indicator. Structured records are extracted effortlessly from the mapped entity.…”