LMLFM: Longitudinal Multi-Level Factorization Machine

Abstract

We consider the problem of learning predictive models from longitudinal data, consisting of irregularly repeated, sparse observations from a set of individuals over time. Such data often exhibit longitudinal correlation (LC) (correlations among observations for each individual over time), cluster correlation (CC) (correlations among individuals that have similar characteristics), or both. These correlations are often accounted for using mixed effects models that include fixed effects and random effects, where the fixed effects capture the regression parameters that are shared by all individuals, whereas random effects capture those parameters that vary across individuals. However, the current state-of-the-art methods are unable to select the most predictive fixed effects and random effects from a large number of variables, while accounting for complex correlation structure in the data and non-linear interactions among the variables. We propose Longitudinal Multi-Level Factorization Machine (LMLFM), to the best of our knowledge, the first model to address these challenges in learning predictive models from longitudinal data. We establish the convergence properties, and analyze the computational complexity, of LMLFM. We present results of experiments with both simulated and real-world longitudinal data which show that LMLFM outperforms the state-of-the-art methods in terms of predictive accuracy, variable selection ability, and scalability to data with large number of variables. The code and supplemental material is available at https://github.com/junjieliang672/LMLFM.

Tensorized LSTM with Adaptive Shared Memory for Learning Trends in Multivariate Time Series

“…There have been intensive interests in developing prediction methods on specific data points for its practical importance (Rangapuram et al 2018;Liang et al 2019). A lot of work in time series forecasting is based on deep learning techniques because of the rapid development of deep neural networks (Dong and De Melo 2018a;Xu et al 2019c;Wang et al 2019b).…”

Predicting fracture outcomes from clinical registry data using artificial intelligence supplemented models for evidence-informed treatment (PRAISE) study protocol

“…Linear and generalized linear mixed models (LMMLasso/GLMMLasso) [ 45 ], longitudinal multi-level factorization machines model (LMLFMM) [ 46 ], longitudinal support vector regression (LS-SVM) [ 47 ] and mixed effects random forest (MERF) [ 48 ] will be used as these machine learning techniques can handle longitudinal data and a large number of potentially correlated features. The inclusion of predictions from Aim 1 will incorporate associated error that potentially compounds further error with their inclusion in the predictive models, so we will use several measures to review model performance.…”

Longitudinal Deep Kernel Gaussian Process Regression

“…In general, the structure of MC can be complex and a priori unknown. Failure to adequately account for the structure of MC in predictive modeling from longitudinal data can lead to misleading statistical inferences [2,3]. It can be non-trivial to choose a suitable correlation structure that reflects the correlations present in the data.…”

“…A popular example of conditional models is the generalized linear mixed-effects models (GLMM) [6]. Despite much work on both marginal and conditional models [7,8,9,3], many of the challenges, especially the choice of correlation structure, and the selection of variables to model random versus fixed effects, and the scalability of the methods remain to be addressed.…”

“…We introduce an EM-like algorithm that combines latent space inducing points and stochastic variational inference for effective training of L-DKGPR. We report results of extensive experiments using both simulated and real-world benchmark longitudinal data that show that L-DKGPR significantly outperforms, by large margins, the popular GLMM, GEE baselines as well as several state-of-the-art non-linear LDA methods [3,12].…”