Combining an Additive and Tree-Based Regression Model Simultaneously: STIMA

Abstract

Additive models and tree-based regression models are two main classes of statistical models used to predict the scores on a continuous response variable. It is known that additive models become very complex in the presence of higher order interaction effects, whereas some tree-based models, such as CART, have problems capturing linear main effects of continuous predictors. To overcome these drawbacks, the regression trunk model has been proposed: a multiple regression model with main effects and a parsimonious amount of higher order interaction effects. The interaction effects can be represented by a small tree: a regression trunk. This article proposes a new algorithmSimultaneous Threshold Interaction Modeling Algorithm (STIMA)-to estimate a regression trunk model that is more general and more efficient than the initial one (RTA) and is implemented in the R-package stima. Results from a simulation study show that the performance of STIMA is satisfactory for sample sizes of 200 or higher. For sample sizes of 300 or higher, the 0.50 SE rule is the best pruning rule for a regression trunk in terms of power and Type I error. For sample sizes of 200, the 0.80 SE rule is recommended. Results from a comparative study of eight regression methods applied to ten benchmark datasets suggest that STIMA and GUIDE are the best performers in terms of cross-validated prediction error. STIMA appeared to be the best method for datasets containing many categorical variables. The characteristics of a regression trunk model are illustrated using the Boston house price dataset.Supplemental materials for this article, including the R-package stima, are available online.

Binary recursive partitioning: Background, methods, and application to psychology

“…As shown in , the regression trunk procedure is easily implemented in R or other programs that can perform both regression and BRP. Alternatively, Dusseldorp, Conversano, and Van Os (in press) present a similar procedure that simultaneously fits trees and regression models, with the trees accounting for interaction effects. This procedure, called STIMA, is implemented in its own R package.…”

“…Other methods that may be of interest to the reader include model‐based BRP (Zeileis, Hothorn, & Hornik, 2008), trees with probabilistic splits (Murthy, 1998; Yuan & Shaw, 1995), and, as mentioned previously, the STIMA procedure (Dusseldorp et al , in press). Finally, in a regression context, nonlinear optimal scaling methods (e.g., van der Kooij, Meulman, & Heiser, 2006) automatically transform predictor and response variables (both discrete and continuous) to optimize the regression‐fitting criterion (e.g., sum of squared error).…”

An analysis of moderators in the COMBINE study: Identifying subgroups of patients who benefit from acamprosate

“…Of the methods presented in this paper only the SIDESscreen approach can be directly applied to continuous outcomes. There are other methods that have been recently developed that can be used for continuous (Su et al, 2009; Dusseldorp et al, 2010; and Dusseldorp and Mechelen, 2014) and censored continuous outcomes (Negassa et al, 2005 and Loh et al, 2014). …”

A modified classification tree method for personalized medicine decisions

“…The approaches proposed by Zhang et al [29] and Foster et al [9] focus on binary outcomes, Su et al [26], Dusseldorp et al [6] and Dusseldorp and Van Mechelen [7] develop interaction tree methods for continuous outcomes, and the approaches of Negassa et al [23] and Loh et al [19] are appropriate for censored continuous outcomes. The SIDES and SIDESscreen approaches [17, 18] can be used with either binary or continuous outcomes.…”