A Further Consideration in the Application of an Analysis‐of‐variance Model for the Intrasubject Replication Design

Kratochwill, Thomas R.; Alden, Kieran; Demuth, D.; Dawson, Daniel G.; Panicucci, Carol; Arntson, Paul; McMurray, N.; Hempstead, Julie E.; Levin, Joel R.

doi:10.1901/jaba.1974.7-629

Cited by 60 publications

(26 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The use of OLS regression for analyzing multiple-baseline data has been advocated (Huitema & McKean, 1998), but the use of OLS regression methods has also raised concerns, because the errors in the model are assumed to be independent. Many have argued that errors closer in time may be more similar to each other than independently selected errors, and, therefore, the errors may be positively autocorrelated instead of independent (Kratochwill et al, 1974;Matyas & Greenwood, 1997). Furthermore, positive autocorrelation impacts the statistical inferences such that there is a greater chance of Type I errors (Greenwood & Matyas, 1990;Toothaker, Banz, Noble, Camp, & Davis, 1983), which in turn implies that 95% confidence intervals would contain the actual effect less than 95% of the time.…”

Section: Ols Regressionmentioning

confidence: 99%

Estimating individual treatment effects from multiple-baseline data: A Monte Carlo study of multilevel-modeling approaches

Ferron

Farmer

Owens

2010

Behavior Research Methods

View full text Add to dashboard Cite

While conducting intervention research, researchers and practitioners are often interested in how the intervention functions not only at the group level, but also at the individual level. One way to examine individual treatment effects is through multiple-baseline studies analyzed with multilevel modeling. This analysis allows for the construction of confidence intervals, which are strongly recommended in the reporting guidelines of the American Psychological Association. The purpose of this study was to examine the accuracy of confidence intervals of individual treatment effects obtained from multilevel modeling of multiple-baseline data. Monte Carlo methods were used to examine performance across conditions varying in the number of participants, the number of observations per participant, and the dependency of errors. The accuracy of the confidence intervals depended on the method used, with the greatest accuracy being obtained when multilevel modeling was coupled with the Kenward-Roger method of estimating degrees of freedom. J. M. Ferron, ferron@usf.edu

show abstract

Section: Ols Regressionmentioning

confidence: 99%

Estimating individual treatment effects from multiple-baseline data: A Monte Carlo study of multilevel-modeling approaches

Ferron

Farmer

Owens

2010

Behavior Research Methods

View full text Add to dashboard Cite

show abstract

“…In the early seventies, several ANOVA-based procedures were suggested (Gentile, Roden, & Klein, 1972;Shine & Bower, 1971), and later ordinary least squares (OLS) regression procedures were illustrated (Center, Skiba, & Casey, 1985-1986Huitema & McKean, 1998). Concerns with these general linear model (GLM) based approaches have been expressed because they assume that the errors in the statistical model are independent (e.g., Kratochwill et al, 1974;McKnight, McKean, & Huitema, 2000), as opposed to autocorrelated.…”

Section: Statistical Methods For Single-case Studymentioning

confidence: 99%

Multilevel models for multiple-baseline data: modeling across-participant variation in autocorrelation and residual variance

Baek

Ferron

2012

Behav Res

View full text Add to dashboard Cite

Multilevel models (MLM) have been used as a method for analyzing multiple-baseline single-case data. However, some concerns can be raised because the models that have been used assume that the Level-1 error covariance matrix is the same for all participants. The purpose of this study was to extend the application of MLM of singlecase data in order to accommodate across-participant variation in the Level-1 residual variance and autocorrelation. This more general model was then used in the analysis of single-case data sets to illustrate the method, to estimate the degree to which the autocorrelation and residual variances differed across participants, and to examine whether inferences about treatment effects were sensitive to whether or not the Level-1 error covariance matrix was allowed to vary across participants. The results from the analyses of five published studies showed that when the Level-1 error covariance matrix was allowed to vary across participants, some relatively large differences in autocorrelation estimates and error variance estimates emerged. The changes in modeling the variance structure did not change the conclusions about which fixed effects were statistically significant in most of the studies, but there was one exception. The fit indices did not consistently support selecting either the more complex covariance structure, which allowed the covariance parameters to vary across participants, or the simpler covariance structure. Given the uncertainty in model specification that may arise when modeling single-case data, researchers should consider conducting sensitivity analyses to examine the degree to which their conclusions are sensitive to modeling choices.

show abstract

“…We will now demonstrate how this method can be applied to derive RTI CIs for ES measures in SCEs, a setting in which the random sampling and distributional assumptions have traditionally been contested (Hartmann, 1974;Houle, 2009;Kratochwill et al, 1974). Note that in the case of an SCE, the repeated measurements are the experimental units whereas in a between-subject design the experimental units refer to the different subjects.…”

Section: Nonparametric Cis For Single-case Designsmentioning

confidence: 99%

Confidence intervals for single-case effect size measures based on randomization test inversion

et al. 2016

View full text Add to dashboard Cite

In the current paper, we present a method to construct nonparametric confidence intervals (CIs) for single-case effect size measures in the context of various single-case designs. We use the relationship between a two-sided statistical hypothesis test at significance level α and a 100 (1 -α) % two-sided CI to construct CIs for any effect size measure θ that contain all point null hypothesis θ values that cannot be rejected by the hypothesis test at significance level α. This method of hypothesis test inversion (HTI) can be employed using a randomization test as the statistical hypothesis test in order to construct a nonparametric CI for θ. We will refer to this procedure as randomization test inversion (RTI). We illustrate RTI in a situation in which θ is the unstandardized and the standardized difference in means between two treatments in a completely randomized single-case design. Additionally, we demonstrate how RTI can be extended to other types of single-case designs. Finally, we discuss a few challenges for RTI as well as possibilities when using the method with other effect size measures, such as rank-based nonoverlap indices. Supplementary to this paper, we provide easy-to-use R code, which allows the user to construct nonparametric CIs according to the proposed method.

show abstract

A Further Consideration in the Application of an Analysis‐of‐variance Model for the Intrasubject Replication Design

Cited by 60 publications

References 6 publications

Estimating individual treatment effects from multiple-baseline data: A Monte Carlo study of multilevel-modeling approaches

Estimating individual treatment effects from multiple-baseline data: A Monte Carlo study of multilevel-modeling approaches

Multilevel models for multiple-baseline data: modeling across-participant variation in autocorrelation and residual variance

Confidence intervals for single-case effect size measures based on randomization test inversion

Contact Info

Product

Resources

About