Permutation‐based inference for the AUC: A unified approach for continuous and discontinuous data

Abstract

We investigate rank-based studentized permutation methods for the nonparametric Behrens-Fisher problem, that is, inference methods for the area under the ROC curve. We hereby prove that the studentized permutation distribution of the Brunner-Munzel rank statistic is asymptotically standard normal, even under the alternative. Thus, incidentally providing the hitherto missing theoretical foundation for the Neubert and Brunner studentized permutation test. In particular, we do not only show its consistency, but also that confidence intervals for the underlying treatment effects can be computed by inverting this permutation test. In addition, we derive permutation-based range-preserving confidence intervals. Extensive simulation studies show that the permutation-based confidence intervals appear to maintain the preassigned coverage probability quite accurately (even for rather small sample sizes). For a convenient application of the proposed methods, a freely available software package for the statistical software R has been developed. A real data example illustrates the application.

Simultaneous inference for factorial multireader diagnostic trials

“…This type of approximation may be improved by exploring resampling techniques. 34 Furthermore, the assumption of crossed factors implies that the same readers diagnose with all diagnostic modalities m. In case of a nested design, different readers are used for investigating and comparing the accuracies of the methods-and the numbers of readers within each modality level m may be different. Let R m denote the numbers of readers being nested under modality combination m, where m = 1, … , M. Then, the diagnosis of subject k obtained from reader r(m) using modality m can be written as X (m,r(m)) ik .…”

Multiplication‐combination tests for incomplete paired data

“…Again, T BM is asymptotically standard normal under and Brunner and Munzel proposed a t ‐approximation for smaller sample sizes. Moreover, the latter was slightly enhanced by a studentized permutation approach leading to the two‐sided permuted Brunner‐Munzel test , where is the corresponding (1 − α 2 /2)‐ permutation quantile . The MCT for testing the hypothesis is thus given by φ ( p ) = φ KP · φ NB .…”

Rank‐based permutation approaches for non‐parametric factorial designs

“…Following the ideas of Neuhaus (), Janssen (, ), Neubert and Brunner (), Pauly (), Konietschke and Pauly (), Omelka and Pauly (), Chung and Romano (, ), Pauly et al . (), Pauly, Asendorf, and Konietschke (), and Smaga (), this problem can be solved by studentizing the statistic . However, this is already achieved by the WTS in equation (2.2), where the inner Moore‐Penrose inverse acts as a multivariate studentization in this quadratic form.…”