Simultaneous confidence intervals for ranks using the partitioning principle

Abstract

We consider the problem of constructing simultaneous confidence intervals (CIs) for the ranks of n means based on their estimates together with the (known) standard errors of those estimates. We present a generic method based on the partitioning principle in which the parameter space is partitioned into disjoint subsets and then each one of them is tested at level α. The resulting CIs have then a simultaneous coverage of 1 − α. We show that any procedure which produces simultaneous CIs for ranks can be written as a partitioning procedure. We present a first example where we test the partitions using the likelihood ratio (LR) test. Then, in a second example we show that a recently proposed method for simultaneous CIs for ranks using Tukey's honest significant difference test has an equivalent procedure based on the partitioning principle. By embedding these two methods inside our generic partitioning procedure, we obtain improved variants. We illustrate the performance of these methods through simulations and real data analysis on hotel ratings. While the novel method that uses the LR test and its variant produce shorter CIs when the number of means is small, the Tukey-based method and its variant produce shorter CIs when the number of means is high.

An Empirical Bayes Method for Differential Expression Analysis of Single Cells with Deep Generative Models

“…Since the development of powerful machine learning systems such as gpt-4, the accuracy of the annotations that these systems produce has started to approach that of humans [3], giving substantial credence to autoevaluation as an alternative to human evaluations. However, standard autoevaluation methods are generally ad hoc, and suffer from bias from the synthetic data; meanwhile, classical solutions for generating confidence intervals, such as rank-sets [9], cannot take advantage of the AI-generated data.…”

Bayesian Inferences on Uncertain Ranks and Orderings: Application to Ranking Players and Lineups

“…In the frequentist framework, some theoretical contributions focus on studying asymptotic conditions under which point-wise confidence intervals of population ranks have the claimed coverage probability (Xie et al, 2009), and under which ranking estimates converge to the truth as the number of parameters and sample size increase to infinity (Hall and Miller, 2010). Other contributions aim to define confidence regions for rankings using multiple confidence intervals or hypothesis tests for the parameters while controlling the family-wise error rate (Wright et al, 2018;Klein et al, 2020;Mohamad et al, 2021). However, controlling the family-wise error rate can be challenging and often leads to regions that are so wide as to be practically meaningless.…”