Asymptotically optimal, sequential, multiple testing procedures with prior information on the number of signals

Song, Yanglei; Fellouris, Georgios

doi:10.1214/17-ejs1223

Cited by 21 publications

(34 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is known [39, p. 278-280] that when θ j n is selected to be the Maximum Likelihood estimator (MLE) of θ j , condition (35) is satisfied when testing a normal mean with unknown variance, as well as when testing the coefficient of a first-order autoregressive model. In Appendix E we further show that condition (35) is satisfied when (i) the data in each stream are i.i.d. with some multi-parameter exponential family distribution, and (ii) the null and the alternative parameter spaces are compact.…”

Section: 2mentioning

confidence: 99%

“…observations, an asymptotic lower bound was obtained in [35] for the optimal expected sample size (ESS) as the error probabilities go to 0, and was shown to be attained, under any signal configuration, by several existing procedures. However, the results in [35] do not extend to generalized error metrics, since the technique for the proof of the asymptotic lower bound requires that the probability of not identifying the correct subset of signals goes to 0. Further, as we shall see, existing procedures fail to be asymptotically optimal, in general, under generalized error metrics.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Sequential multiple testing with generalized error control: An asymptotic optimality theory

Song¹,

Fellouris²

2019

Ann. Statist.

Self Cite

View full text Add to dashboard Cite

The sequential multiple testing problem is considered under two generalized error metrics. Under the first one, the probability of at least k mistakes, of any kind, is controlled. Under the second, the probabilities of at least k1 false positives and at least k2 false negatives are simultaneously controlled. For each formulation, the optimal expected sample size is characterized, to a first-order asymptotic approximation as the error probabilities go to 0, and a novel multiple testing procedure is proposed and shown to be asymptotically efficient under every signal configuration. These results are established when the data streams for the various hypotheses are independent and each local log-likelihood ratio statistic satisfies a certain Strong Law of Large Numbers. In the special case of i.i.d. observations in each stream, the gains of the proposed sequential procedures over fixed-sample size schemes are quantified. MSC 2010 subject classifications: Primary 62L10

show abstract

Section: 2mentioning

confidence: 99%

mentioning

confidence: 99%

Sequential multiple testing with generalized error control: An asymptotic optimality theory

Song¹,

Fellouris²

2019

Ann. Statist.

Self Cite

View full text Add to dashboard Cite

show abstract

“…When dealing with composite hypotheses, computing optimal policies is intractable even for the single process case. For tractability, a commonly adopted performance measure is asymptotic optimality in terms of minimizing the detection time as the error probability approaches zero (see, for example, classic and recent results in [8]- [10], [13]- [23]). The focus of this paper is thus on asymptotically optimal strategies with low computational complexity.…”

Section: A Main Resultsmentioning

confidence: 99%

Searching for Anomalies Over Composite Hypotheses

Hemo

Gafni

Cohen

et al. 2020

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

The problem of detecting anomalies in multiple processes is considered. We consider a composite hypothesis case, in which the measurements drawn when observing a process follow a common distribution with an unknown parameter (vector), whose value lies in normal or abnormal parameter spaces, depending on its state. The objective is a sequential search strategy that minimizes the expected detection time subject to an error probability constraint. We develop a deterministic search algorithm with the following desired properties. First, when no additional side information on the process states is known, the proposed algorithm is asymptotically optimal in terms of minimizing the detection delay as the error probability approaches zero. Second, when the parameter value under the null hypothesis is known and equal for all normal processes, the proposed algorithm is asymptotically optimal as well, with better detection time determined by the true null state. Third, when the parameter value under the null hypothesis is unknown, but is known to be equal for all normal processes, the proposed algorithm is consistent in terms of achieving error probability that decays to zero with the detection delay. Finally, an explicit upper bound on the error probability under the proposed algorithm is established for the Bar Hemo, Tomer Gafni and Kobi Cohen are with the

show abstract

“…A large body of literature on sequential tests with two hypotheses has been developed, a partial list of which includes [27,34,50]. Sequential testing with more than two hypotheses and sequential multiple testing have been extensively studied in recent decades (see, e.g., [21,22,43,53,62]). For a comprehensive review on sequential analysis, we refer the readers to the surveys and books [29,35,52,54] and references therein.…”

Section: Main Contributionmentioning

confidence: 99%

Asymptotically Optimal Sequential Design for Rank Aggregation

Chen¹,

Chen²,

2022

Mathematics of OR

View full text Add to dashboard Cite

A sequential design problem for rank aggregation is commonly encountered in psychology, politics, marketing, sports, etc. In this problem, a decision maker is responsible for ranking K items by sequentially collecting noisy pairwise comparisons from judges. The decision maker needs to choose a pair of items for comparison in each step, decide when to stop data collection, and make a final decision after stopping based on a sequential flow of information. Because of the complex ranking structure, existing sequential analysis methods are not suitable. In this paper, we formulate the problem under a Bayesian decision framework and propose sequential procedures that are asymptotically optimal. These procedures achieve asymptotic optimality by seeking a balance between exploration (i.e., finding the most indistinguishable pair of items) and exploitation (i.e., comparing the most indistinguishable pair based on the current information). New analytical tools are developed for proving the asymptotic results, combining advanced change of measure techniques for handling the level crossing of likelihood ratios and classic large deviation results for martingales, which are of separate theoretical interest in solving complex sequential design problems. A mirror-descent algorithm is developed for the computation of the proposed sequential procedures.

show abstract

Asymptotically optimal, sequential, multiple testing procedures with prior information on the number of signals

Cited by 21 publications

References 27 publications

Sequential multiple testing with generalized error control: An asymptotic optimality theory

Sequential multiple testing with generalized error control: An asymptotic optimality theory

Searching for Anomalies Over Composite Hypotheses

Asymptotically Optimal Sequential Design for Rank Aggregation

Contact Info

Product

Resources

About