Mixed effect machine learning: A framework for predicting longitudinal change in hemoglobin A1c

Abstract

Accurate and reliable prediction of clinical progression over time has the potential to improve the outcomes of chronic disease. The classical approach to analyzing longitudinal data is to use (generalized) linear mixed-effect models (GLMM). However, linear parametric models are predicated on assumptions, which are often difficult to verify. In contrast, data-driven machine learning methods can be applied to derive insight from the raw data without a priori assumptions. However, the underlying theory of most machine learning algorithms assume that the data is independent and identically distributed, making them inefficient for longitudinal supervised learning. In this study, we formulate an analytic framework, which integrates the random-effects structure of GLMM into non-linear machine learning models capable of exploiting temporal heterogeneous effects, sparse and varying-length patient characteristics inherent in longitudinal data. We applied the derived mixed-effect machine learning (MEml) framework to predict longitudinal change in glycemic control measured by hemoglobin A1c (HbA1c) among well controlled adults with type 2 diabetes. Results show that MEml is competitive with traditional GLMM, but substantially outperformed standard machine learning models that do not account for random-effects. Specifically, the accuracy of MEml in predicting glycemic change at the 1st, 2nd, 3rd, and 4th clinical visits in advanced was 1.04, 1.08, 1.11, and 1.14 times that of the gradient boosted model respectively, with similar results for the other methods. To further demonstrate the general applicability of MEml, a series of experiments were performed using real publicly available and synthetic data sets for accuracy and robustness. These experiments reinforced the superiority of MEml over the other methods. Overall, results from this study highlight the importance of modeling random-effects in machine learning approaches based on longitudinal data. Our MEml method is highly resistant to correlated data, readily accounts for random-effects, and predicts change of a longitudinal clinical outcome in real-world clinical settings with high accuracy.

Association between exposure to airborne pollutants and COVID-19 in Los Angeles, United States with ensemble-based dynamic emission model

“…For instance, equations (1) , (2) ), delineate our assumption for the linear dependence of COVID-19 cases and deaths, respectively. Although the knowledge is obtained from existing studies, the use of such assumptions is highly discouraged in association analysis ( Ngufor et al, 2019 ). Nevertheless, data-driven machine learning approaches have the potential to obtain associations, both linear and non-linear, without facilitating the model with linear apriori assumptions.…”

“…We begin by modeling the pollutants with Generalized Linear Models (GLMs), a classical technique in ecological modeling. However, the GLM has well-established limitations: apriori assumptions ( Ngufor et al, 2019 ), inability to handle multicollinearity ( Chen et al, 2018 ; Dastoorpoor et al, 2019 ; Phosri et al, 2019 ), and quantifying differential county-level (or city-level) effects as fixed effects ( Zheng et al, 2020 ). We address these limitations by devising a novel modeling approach, Ensemble-based Dynamic Emission Modeling (EDEM), at the intersection of network science and machine learning.…”

A translucent box: interpretable machine learning in ecology

“…However, combining random effects with nonparametric, non‐statistical models is more difficult. While these models are starting to be developed (Sela and Simonoff 2011, Eo and Cho 2014, Hajjem et al 2014, 2017, Miller et al 2017, Ngufor et al 2019), they are not available in R packages, are only implemented for a small subset of non‐statistical models, and do not necessarily benefit from the computational improvements implemented in the most up‐to‐date packages (Wright and Ziegler 2015, Chen and Guestrin 2016). Therefore, generic methods for handling random effects, that can be used with any machine learning algorithm, are useful.…”

Artificial Intelligence and Machine Learning in Endocrinology and Metabolism: The Dawn of a New Era

“…The above examples reinforce the promising role of AI/ML in diagnosis and management of endocrine disorders which, in several instances, can outperform skilled physicians, minimize resource use and allocation, and yield tangible benefits by supporting physicians and accelerating clinical decision-making (7, 16). Despite the substantial evidence for the ability of AI/ML to deliver cost-effective healthcare and improve patient outcomes, medicine has trailed behind other scientific fields in implementing these techniques into practice (21). Potential hurdles include the longitudinal nature of variations in human disease, inadequacies in the quality and reliability, heterogeneity of healthcare data, personal data confidentiality, need for informed consent from patients, requirement of supportive policies and efficient business models, unpredictable reimbursement, and increasing necessity for data sharing (21, 22).…”

“…Despite the substantial evidence for the ability of AI/ML to deliver cost-effective healthcare and improve patient outcomes, medicine has trailed behind other scientific fields in implementing these techniques into practice (21). Potential hurdles include the longitudinal nature of variations in human disease, inadequacies in the quality and reliability, heterogeneity of healthcare data, personal data confidentiality, need for informed consent from patients, requirement of supportive policies and efficient business models, unpredictable reimbursement, and increasing necessity for data sharing (21, 22). The so-called “digital biomarkers” that are obtained through big data analyses performed using AI/ML techniques are not readily interpretable clinically, in the sense, even if a certain newer AI/ML algorithm has been shown to be superior to older techniques in certain population cohorts; its implementation in clinical practice across more diverse populations might not necessarily result in better diagnosis or outcome; and could potentially even lead to over-diagnosis and over-treatment in certain patient cohorts (23).…”