Simultaneous confidence intervals for ranks with application to ranking institutions

Abstract

When a ranking of institutions such as medical centers or universities is based on an indicator provided with a standard error, confidence intervals should be calculated to assess the quality of these ranks. We consider the problem of constructing simultaneous confidence intervals (CIs) for the ranks of centers based on an observed sample. We present a novel method based on Tukey's honest significant difference test (HSD) which is the first method to produce valid simultaneous CIs for ranks. Moreover, we introduce a new variant of Tukey's HSD based on the sequential rejection principle. The new algorithm ensures familywise error control, and produces simultaneous confidence intervals for the ranks uniformly shorter than those provided by Tukey's HSD for the same level of significance. We illustrate the method through both simulations and real data analysis from 64 hospitals in the Netherlands. Software for our new methods is available online in package ICRanks downloadable from CRAN. Supplementary materials include supplementary R code for the simulations and proofs of the propositions presented in this paper.

Simultaneous confidence intervals for ranks using the partitioning principle

“…Simulation studies show that the method of [1] based on Tukey's HSD is more powerful than the one based on the LR test especially when the number of means is high while the converse happens when the number of means is small. The improved versions of these methods are shown to be computationally feasible only up to n = 10.…”

“…Methods for simultaneous CIs for ranks are proposed only by [47,1,25]. The method of [47] uses the parametric bootstrap to construct CIs for ranks and then Monte-Carlo simulations to estimate the simultaneous coverage.…”

“…The method of [47] uses the parametric bootstrap to construct CIs for ranks and then Monte-Carlo simulations to estimate the simultaneous coverage. In [1], the authors show that the method of [47] is anticonservative and propose a new method based on Tukey's honest significant difference (HSD) test which ensures that the simultaneous confidence level is at least 1 − α. The method of [25] creates simultaneous CIs for ranks starting from simultaneous CIs for the means which result in less powerful results than the method of [1].…”

“…Although the complexity of the procedure is very high, we show some shortcuts which allow the method to be feasible with up to 30 − 40 means with a regular computer. Second, we show that the method of [1] based on Tukey's HSD can be written as a partitioning procedure by giving an explicit local test for the partitions. Using our generic partitioning procedure, we present two variants of the partitioning procedure that uses the LR test and the Tukey-based method which uniformly improve their corresponding methods.…”

“…In Section 5, we present the likelihood ratio (LR) test and use it as a first example of a local test. In Section 6, we revisit the Tukey-based method of [1] and give an equivalent procedure using the partitioning principle. Finally, in Section 7, we compare the results of the LR test and the Tukey-based method on simulated and real data samples.…”