Single-Case Intervention Research Design Standards

Abstract

In an effort to responsibly incorporate evidence based on single-case designs (SCDs) into the What Works Clearinghouse (WWC) evidence base, the WWC assembled a panel of individuals with expertise in quantitative methods and SCD methodology to draft SCD standards. In this article, the panel provides an overview of the SCD standards recommended by the panel (henceforth referred to as the Standards) and adopted in Version 1.0 of the WWC's official pilot standards. The Standards are sequentially applied to research studies that incorporate SCDs. The design standards focus on the methodological soundness of SCDs, whereby reviewers assign the categories of Meets Standards, Meets Standards With Reservations, and Does Not Meet Standards to each study. Evidence criteria focus on the credibility of the reported evidence, whereby the outcome measures that meet the design standards (with or without reservations) are examined by reviewers trained in visual analysis and categorized as demonstrating Strong Evidence, Moderate Evidence, or No Evidence. An illustration of an actual research application of the Standards is provided. Issues that the panel did not address are presented as priorities for future consideration. Implications for research and the evidence-based practice movement in psychology and education are discussed. The WWC's Version 1.0 SCD standards are currently being piloted in systematic reviews conducted by the WWC. This document reflects the initial standards recommended by the authors as well as the underlying rationale for those standards. It should be noted that the WWC may revise the Version 1.0 standards based on the results of the pilot; future versions of the WWC standards can be found at http://www.whatworks.ed.gov.

Implementing and evaluating early intervention for children with autism: Where are the gaps and what should we do?

“…Although there is no shortage of proof-ofconcept and pilot evaluations in ASD early intervention, guidelines on what can be learned from such studies [e.g., Craig et al, 2008;Kratochwill et al, 2010] are not utilized consistently. In particular, pilot studies in this field often focus on drawing inference regarding evidence for the treatment benefits (even though the study was not designed or adequately powered to provide this information), and neglect critical information on treatment feasibility, and social validity/acceptability to key stakeholders (a critical step for moving towards tests of effectiveness).…”

Optimizing behavioral health interventions with single-case designs: from development to dissemination

“…The durations of the baseline conditions are staggered for each participant, and the dependent variable increases when, and only when, the independent variable is introduced for all participants. The SCD standards requires at least six phases (i.e., three baseline and three treatment) with at least five data points per phase [31]. Figure 2 suggests reliable increases in ESSAY/OPINION PIECE behavior and that the treatment was responsible for these changes.…”

“…• The dependent variable is socially important • The magnitude of change in the dependent variable resulting from the intervention is socially important • The methods are acceptable to the participant ESSAY/OPINION PIECE case research to inform policy and practice (hereafter referred to as the SCD standards) [31]. All of the designs listed in Table 1 entail a baseline period of observation.…”

“…Using only two conditions, such as a pre-post design, is not considered sufficient to demonstrate experimental control because other sources of influence on behavior cannot be ruled out [31,32]. For example, a smoking cessation intervention could coincide with a price increase in cigarettes.…”

“…According to the SCD standards, there must be a minimum of three replications to support conclusions about experimental control and thus causation. Also, according to the SCD standards, there must be at least three and preferably five data points per phase to allow the researcher to evaluate stability and experimental effects [31].…”