Multilevel models for multiple-baseline data: modeling across-participant variation in autocorrelation and residual variance

Abstract

Multilevel models (MLM) have been used as a method for analyzing multiple-baseline single-case data. However, some concerns can be raised because the models that have been used assume that the Level-1 error covariance matrix is the same for all participants. The purpose of this study was to extend the application of MLM of singlecase data in order to accommodate across-participant variation in the Level-1 residual variance and autocorrelation. This more general model was then used in the analysis of single-case data sets to illustrate the method, to estimate the degree to which the autocorrelation and residual variances differed across participants, and to examine whether inferences about treatment effects were sensitive to whether or not the Level-1 error covariance matrix was allowed to vary across participants. The results from the analyses of five published studies showed that when the Level-1 error covariance matrix was allowed to vary across participants, some relatively large differences in autocorrelation estimates and error variance estimates emerged. The changes in modeling the variance structure did not change the conclusions about which fixed effects were statistically significant in most of the studies, but there was one exception. The fit indices did not consistently support selecting either the more complex covariance structure, which allowed the covariance parameters to vary across participants, or the simpler covariance structure. Given the uncertainty in model specification that may arise when modeling single-case data, researchers should consider conducting sensitivity analyses to examine the degree to which their conclusions are sensitive to modeling choices.

Applying mixed‐effects modeling to single‐subject designs: An introduction

“…More recent developments in longitudinal statistical analyses, namely mixed-effects modeling, can aptly address the concerns of data compression both across individuals and across time (Hox, Moerbeek, & Van de Schoot, 2017). Although mixed-effects modeling (also known as hierarchical or multilevel modeling) has grown in popularity in some corners of behavior analysis (e.g., delay discounting; Friedel, DeHart, Frye, Rung, & Odum, 2016;Kirkpatrick, Marshall, Steele, & Peterson, 2018;Young, 2017Young, , 2018b, the application to single-subject designs is limited (Baek & Ferron, 2013;Nugent, 1996). Mixedeffects models are designed to handle certain violations of assumptions in the normal regression model (Boisgontier & Cheval, 2016) that arise when analyzing single-subject design data, ultimately providing more accurate and less biased predictions of the underlying trends in the data.…”

The use of multilevel analysis for integrating single-case experimental design results within a study and across studies

“…One of the advantages of a multilevel modelling framework is that the researcher does not have to assume that the errors in a SCED are independent. Rather, a variety of error structures can be assumed: the first-order autoregressive structure, a 2-band toeplitz structure (or first-order moving average structure) and the more general toeplitz structure (Baek & Ferron, 2013;Ferron, Bell, Hess, Rendina-Gobioff, & Hibbard, 2009). …”

“…The covariance structure is usually assumed to be homogeneous within cases and across cases, but this is not always the case (Baek & Ferron, 2013). Therefore the multilevel model can be extended by modelling heterogeneous between-case variance (i.e., each participant has a different error variance) 596 BAEK ET AL.…”

Using Visual Analysis to Evaluate and Refine Multilevel Models of Single-Case Studies

“…Multilevel models allow researchers to (a) estimate an effect for each participant, (b) index the variation in the effect between participants within a study, (c) index the variation in the average effects between studies, and (d) model the variation in effects as a function of participant and/or study characteristics. Multilevel models also allow researchers to (e) estimate the effect at a particular point in time (e.g., after three intervention sessions), (f) quantify the changes in the effect over the course of the intervention, and (g) examine the degree to which changes in the effect during intervention may vary across participants and studies as a 565367S EDXXX10.1177/0022466914565367The Journal of Special EducationBaek et al (Ferron, Bell, Hess, Rendina-Gobioff, & Hibbard, 2009), changes in variability between cases or phases (Baek & Ferron, 2013), and nonnormally distributed outcomes, such as counts (Shadish, Kyse, & Rindskopf, 2013).…”