Application of the percentage of non-overlapping data (PND) in systematic reviews and meta-analyses: A systematic review of reporting characteristics

Schlosser, Ralf W.; Lee, David L.; Wendt, Oliver

doi:10.1080/17489530802505412

Cited by 75 publications

(43 citation statements)

References 52 publications

(90 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Three different ES measures were used as the test statistic in the RT: the PND, the NAP, and the MD. The main reason for including the PND in this simulation study is that it is the most widely used NES (Maggin et al, 2011;Schlosser et al, 2008). As such, we believe it is of great importance to investigate PND's usability in statistical inferences.…”

Section: Design Matrix Of the Simulation Studymentioning

confidence: 99%

“…The PND is the earliest published NES measure and the most widely used one (Maggin et al, 2011;Schlosser, Lee & Wendt, 2008). The PND is calculated as the percentage of data points from the treatment condition that exceeds the single highest data point from the control condition (assuming that the treatment is intended to increase the dependent variable).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The conditional power of randomization tests for single-case effect sizes in designs with randomized treatment order: A Monte Carlo simulation study

2017

View full text Add to dashboard Cite

The conditional power (CP) of the randomization test (RT) was investigated in a simulation study in which three different single-case effect size (ES) measures were used as the test statistic: the mean difference (MD), the Percentage of Nonoverlapping Data (PND), and the Nonoverlap of All Pairs (NAP). Furthermore, we studied the effect of the experimental design on the RT's CP for three different single-case designs with rapid treatment alternation: the completely randomized design (CRD), the randomized block design (RBD), and the restricted randomized alternation design (RRAD). As a third goal, we evaluated the CP of the RT for three types of simulated data: data generated from a standard normal distribution, data generated from a uniform distribution, and data generated from a first-order autoregressive Gaussian process. The results showed that the MD and NAP perform very similar in terms of CP whereas the PND performs substantially worse. Furthermore, the RRAD yielded marginally higher power in the RT, followed by the CRD and then the RBD. Finally, the power of the RT was almost unaffected by the type of simulated data. Based on the results of the simulation study, we recommend at least 20 measurement occasions for single-case designs with randomized treatment order that are to be evaluated with an RT using a 5% significance level. Furthermore, we do not recommend the use of the PND because of its low power in the RT.

show abstract

Section: Design Matrix Of the Simulation Studymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

The conditional power of randomization tests for single-case effect sizes in designs with randomized treatment order: A Monte Carlo simulation study

2017

View full text Add to dashboard Cite

show abstract

“…The integration of single-case studies is especially relevant given the usual lack of random sampling of participants and as generalization is based on replication (Sidman, 1960). Among the possibilities for integrating data, there have already been studies using simple (Schlosser, Lee, & Wendt, 2008) or weighted averages of the primary indicators (Schneider, Goldstein, & Parker, 2008). A proposal for weighting has also been made for primary indicators for which the standard error is unknown (Manolov & Solanas, 2012).…”

Section: Quantitative Analysismentioning

confidence: 99%

A comparison of mean phase difference and generalized least squares for analyzing single-case data

Manolov

Solanas

2013

Journal of School Psychology

View full text Add to dashboard Cite

The present study focuses on single-case data analysis and specifically on two procedures for quantifying differences between baseline and treatment measurementsThe first technique tested is based on generalized least squares regression analysis and is compared to a proposed non-regression technique, which allows obtaining similar information. The comparison is carried out in the context of generated data representing a variety of patterns (i.e., independent measurements, different serial dependence underlying processes, constant or phase-specific autocorrelation and data variability, different types of trend, and slope and level change). The results suggest that the two techniques perform adequately for a wide range of conditions and researchers can use both of them with certain guarantees. The regression-based procedure offers more efficient estimates, whereas the proposed non-regression procedure is more sensitive to intervention effects. Considering current and previous findings, some tentative recommendations are offered to applied researchers in order to help choosing among the plurality of single-case data analysis techniques.

show abstract

“…Nonetheless, its proponents state that there is no "strong statistical justification" (p. 24) for its use. Finally, using unweighted averages has also been considered and appears to be a common practice (Schlosser, Lee, & Wendt, 2008).…”

Section: Weighting Strategiesmentioning

confidence: 99%

Weighting strategies in the meta-analysis of single-case studies

2014

View full text Add to dashboard Cite

Establishing the evidence base of interventions taking place in areas such as psychology and special education is one of the research aims of single-case designs in conjunction with the aim of improving the well-being of participants in the studies. The scientific criteria for solid evidence focus on the internal and external validity of the studies, and for both types of validity, replicating studies and integrating the results of these replications (i.e., meta-analyzing) is crucial. In the present study we deal with one of the aspects of meta-analysis, namely the weighting strategy used when computing an average effect size across studies. Several weighting strategies suggested for single-case designs are discussed and compared in the context of both simulated and real-life data. The results indicated that there are no major differences between the strategies and, thus we consider that it is important to choose weights with a sound statistical and methodological basis, while scientific parsimony is another relevant criterion. More empirical research and conceptual discussion are warranted regarding the optimal weighting strategy in single-case designs, alongside investigation of the optimal effect size measure in these types of design.Key words: single-case designs; meta-analysis; weight; effect size 3 The evidence-based movement has now been salient for several years in a variety of disciplines including psychology (APA Presidential Task Force on Evidence-Based Practice, 2006), medicine (Sackett, Rosenberg, Gray, Hayness, & Richardson, 1996), and special education . In this context, single-case designs ( Sidman, 1960) and, apart from using random assignment of conditions to measurement times, it is also favored by replication of the behavioral change contiguous with the change in conditions (Kratochwill et al., 2013;Wolery, 2013). On the other hand, replication is also related to generalization (Sidman, 1960), which benefits from research synthesis and metaanalysis. In that sense, the evidence-based movement has also paid attention to the metaanalytical integration of replications or studies on the same topic (Beretvas & Chung, 2008b;Jenson, Clark, Kircher, & Kristjansson, 2007). The quantitative integration is deemed especially useful when moderator variables are included in the meta-analyses (Burns, 2012;Wolery, 2013).Finally, it has been stressed that meta-analysis and the assessment of internal and external 1 We chose the term single-case designs (SCD) in order to be consistent with the labeling used in the articles recently published in this journal by, for instance, Baek and Ferron (2013) ). The present study represents an effort to discuss and obtain evidence regarding the metaanalysis of single-case studies; its focus is on weighting strategies rather than on the effect size measures that summarize the results. In that sense, it should be stressed that we do not advocate here for or against specific procedures for SCD data analysis. We consider that, while the debate on the optimal analytical...

show abstract

Application of the percentage of non-overlapping data (PND) in systematic reviews and meta-analyses: A systematic review of reporting characteristics

Cited by 75 publications

References 52 publications

The conditional power of randomization tests for single-case effect sizes in designs with randomized treatment order: A Monte Carlo simulation study

The conditional power of randomization tests for single-case effect sizes in designs with randomized treatment order: A Monte Carlo simulation study

A comparison of mean phase difference and generalized least squares for analyzing single-case data

Weighting strategies in the meta-analysis of single-case studies

Contact Info

Product

Resources

About