RE-EM trees: a data mining approach for longitudinal and clustered data

Abstract

Longitudinal data refer to the situation where repeated observations are available for each sampled object. Clustered data, where observations are nested in a hierarchical structure within objects (without time necessarily being involved) represent a similar type of situation. Methodologies that take this structure into account allow for the possibilities of systematic differences between objects that are not related to attributes and autocorrelation within objects across time periods. A standard methodology in the statistics literature for this type of data is the mixed effects model, where these differences between objects are represented by so-called "random effects" that are estimated from the data (population-level relationships are termed "fixed effects," together resulting in a mixed effects model). This paper presents a methodology that combines the structure of mixed effects models for longitudinal and clustered data with the flexibility of tree-based estimation methods. We apply the resulting estimation method, called the RE-EM tree, to pricing in online transactions, showing that the RE-EM tree is less sensitive to parametric assumptions and provides improved predictive power compared to linear models with random effects and regression trees without random effects. We also apply it to a smaller data set examining accident fatalities, and show that the RE-EM tree strongly outperforms a tree without random effects while performing comparably to a linear model with random effects. We also perform extensive simulation experiments to show that the estimator improves predictive performance relative to regression trees without random effects and is comparable or superior to using linear models with random effects in more general situations.

Socio-economic status and self-rated health: Are they good predictors of income? An analysis of survey panel data from Turkey

“…The RE-EM model is a regression tree-based model with RE for panel data. [24] This model merges the flexibility of tree-based predictive models with the structure of mixed effects models for panel data. [32] RE-EM trees are low sensitive to parametric assumptions and have good predictive power when compared with linear models with RE and regression trees without RE.…”

“…[23] Longitudinal (panel) data refer to the situation where repeated observations are available for each sampled object. [24] Panel data refers to datasets for a cohort of agents, which may be individuals or aggregated data for an entire city or region, gathered over a period and indexed by both the time and cohort variables. It is a multidimensional time series coming from the continuous observation of cross-sections.…”

Female mountain goats, Oreamnos americanus , associate according to kinship and reproductive status

“…For example, individuals a and b were often in the same groups and so were individuals a and c. As a consequence, individuals b and c were also often in the same groups. Thus, using HWIGs, we constructed two regression trees (De'ath, 2002) with random effects (EM algorithm: Sela & Simonoff, 2009), the first including all females, the second one including only sexually mature females !3 years old. We carried out this second test to verify whether motheredaughter associations remained significant after young females were removed from the data set.…”

“…The random effect multivariate regression tree is a nonparametric dichotomous approach to separate data according to the variable best explaining their distribution. For each new branch, the regression tree renews the calculation and separates data on the basis of the new variable best explaining the separation (Sela & Simonoff, 2009). Dyads were observed over several years, thus we included both members of a dyad as random effects as well as year (2005e2008).…”

A random forest approach to capture genetic effects in the presence of population structure

“…We term this approach mixed RF and show that it flexibly combines the advantages of the standard LMM and the RF. Related methods, combining random effect models with regression trees or RF, have recently been proposed in different contexts 31,32 .…”