Randomization tests: A new gold standard?

Dugard, Pat

doi:10.1016/j.jcbs.2013.10.001

Cited by 31 publications

(26 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These advantages are important because it has been shown that the assumptions underlying parametric tests (e.g., random sampling or specific distributional assumptions) are doubtful in many domains of behavioral research and particularly for single-case research (e.g., Adams & Anthony, 1996;Dugard, 2014;Edgington & Onghena, 2007;Levin et al, 2014;Micceri, 1989). Other advantages of the RT compared to parametric tests are its flexibility with regard to the choice of test statistic and the choice of experimental design (Ferron & Sentovich, 2002;Onghena, 1992;Onghena & Edgington, 2005).…”

Section: Evaluating Treatment Effects In Single-case Experimentsmentioning

confidence: 99%

“…However, parametric procedures such as statistical tests and confidence intervals based on t-and F-distributions are often not appropriate to analyze SCE data because the assumptions underlying these procedures (e.g., random sampling and more specific distributional assumptions) are often violated in many areas of behavioral research and particularly in single-case research (e.g., Adams & Anthony, 1996;Dugard, 2014;Edgington & Onghena, 2007;Levin, Ferron, & Gafurov, 2014;Micceri, 1989). In contrast, nonparametric procedures do not make specific distributional assumptions about the data.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The conditional power of randomization tests for single-case effect sizes in designs with randomized treatment order: A Monte Carlo simulation study

2017

View full text Add to dashboard Cite

The conditional power (CP) of the randomization test (RT) was investigated in a simulation study in which three different single-case effect size (ES) measures were used as the test statistic: the mean difference (MD), the Percentage of Nonoverlapping Data (PND), and the Nonoverlap of All Pairs (NAP). Furthermore, we studied the effect of the experimental design on the RT's CP for three different single-case designs with rapid treatment alternation: the completely randomized design (CRD), the randomized block design (RBD), and the restricted randomized alternation design (RRAD). As a third goal, we evaluated the CP of the RT for three types of simulated data: data generated from a standard normal distribution, data generated from a uniform distribution, and data generated from a first-order autoregressive Gaussian process. The results showed that the MD and NAP perform very similar in terms of CP whereas the PND performs substantially worse. Furthermore, the RRAD yielded marginally higher power in the RT, followed by the CRD and then the RBD. Finally, the power of the RT was almost unaffected by the type of simulated data. Based on the results of the simulation study, we recommend at least 20 measurement occasions for single-case designs with randomized treatment order that are to be evaluated with an RT using a 5% significance level. Furthermore, we do not recommend the use of the PND because of its low power in the RT.

show abstract

Section: Evaluating Treatment Effects In Single-case Experimentsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

The conditional power of randomization tests for single-case effect sizes in designs with randomized treatment order: A Monte Carlo simulation study

2017

View full text Add to dashboard Cite

show abstract

“…We used a Monte Carlo permutation test [8][9][10]17] to analyze possible statistically significant differences between gaze angle data sets. The permutation test is not dependent on as many assumptions on the sample distribution as some other tests such as ANOVA [8], especially as the test sample need not be normally distributed.…”

Section: Discussionmentioning

confidence: 99%

“…The permutation test is not dependent on as many assumptions on the sample distribution as some other tests such as ANOVA [8], especially as the test sample need not be normally distributed. Also, we were using median values as the test statistic, while some other methods can only use the mean.…”

Section: Discussionmentioning

confidence: 99%

Gaze Cueing with a Vibrotactile Headband for a Visual Search Task

2017

View full text Add to dashboard Cite

Augmented attention, assisting the user in noticing important things, is one of the ways human action can be enhanced with technologies. We investigated how vibrotactile stimulation given to the forehead could be used to cue gaze direction. We built a vibrotactile headband with an array of six actuators that presented short, tap-like cues. In the first experiment, the participant was instructed to look at the point on a horizontal line that they thought the vibrotactile cue was pointing to. Analysis of the participant's gaze points showed that for the majority there were statistically significant differences between cues from different actuators. This indicated that the six actuators could successfully direct the participant's gaze to different areas of the visual field. In addition, vibrotactile cueing of gaze direction could be used for directing visual attention and providing navigation cues with wearable headbands. To strengthen our findings, we investigated how effective the vibrotactile stimulation would be to cue gaze direction in a visual search task. Participant's were asked to find a deviant shape (a target) from a display full of simple shapes. The vibrotactile cueing implemented with the headband device was used to inform the participants of the approximate horizontal position of the target in three different experimental conditions. In the most informative condition, six actuators were used to inform the participant of the horizontal area where the target would be found, in the second condition two actuators were used to inform the participant of the target side on the display (left or right), and in the least informative condition no directional information was given. Analysis of the trial completion times showed that there were statistically significant differences between the least informative condition and the two other conditions. However, we did not find significant differences in trial completion times between the two conditions where information of the target location was given. This indicated that while the actuators could successfully direct the participant's attention to different areas of the visual field to help in the search task, the simple approach of just adding actuators and dividing the visual field to more sub-areas did not improve the results. The findings of this study showed that while there is potential in using vibrotactile cueing of gaze direction, more research is needed to fully exploit it.

show abstract

“…However, the validity of statistical inferences based on parametric procedures is frequently doubtful because the assumptions underlying these procedures (e.g., random sampling or specific distributional assumptions) are implausible in many areas of behavioral research, and for single-case research in particular (e.g., Adams & Anthony, 1996;Dugard, 2014;Edgington & Onghena, 2007;Levin, Ferron, & Gafurov, 2014;Micceri, 1989). Therefore, the randomization test (RT) has been proposed as a nonparametric alternative to test statistical hypotheses in randomized SCEs (e.g., Bulté & Onghena, 2008;Edgington, 1967;Onghena & Edgington, 1994Heyvaert & Onghena, 2014;Levin, Ferron & Kratochwill, 2012;Onghena, 1992).…”

Section: Introductionmentioning

confidence: 99%

Confidence intervals for single-case effect size measures based on randomization test inversion

et al. 2016

View full text Add to dashboard Cite

In the current paper, we present a method to construct nonparametric confidence intervals (CIs) for single-case effect size measures in the context of various single-case designs. We use the relationship between a two-sided statistical hypothesis test at significance level α and a 100 (1 -α) % two-sided CI to construct CIs for any effect size measure θ that contain all point null hypothesis θ values that cannot be rejected by the hypothesis test at significance level α. This method of hypothesis test inversion (HTI) can be employed using a randomization test as the statistical hypothesis test in order to construct a nonparametric CI for θ. We will refer to this procedure as randomization test inversion (RTI). We illustrate RTI in a situation in which θ is the unstandardized and the standardized difference in means between two treatments in a completely randomized single-case design. Additionally, we demonstrate how RTI can be extended to other types of single-case designs. Finally, we discuss a few challenges for RTI as well as possibilities when using the method with other effect size measures, such as rank-based nonoverlap indices. Supplementary to this paper, we provide easy-to-use R code, which allows the user to construct nonparametric CIs according to the proposed method.

show abstract

Randomization tests: A new gold standard?

Cited by 31 publications

References 14 publications

The conditional power of randomization tests for single-case effect sizes in designs with randomized treatment order: A Monte Carlo simulation study

The conditional power of randomization tests for single-case effect sizes in designs with randomized treatment order: A Monte Carlo simulation study

Gaze Cueing with a Vibrotactile Headband for a Visual Search Task

Confidence intervals for single-case effect size measures based on randomization test inversion

Contact Info

Product

Resources

About