Competitive analysis of the top-<i>K</i> ranking problem

Chen, Xi; Gopi, Sivakanth; Mao, Jieming; Schneider, Jon

doi:10.1137/1.9781611974782.81

“…Recently, Chen et al [11] focused on computing the smallest k elements given r independent noisy comparisons between each pair of elements. For this problem, in a more general error model, they provide a tight algorithm that requires at most O( √ n polylog n) times as many samples as the best possible algorithm that achieves the same success probability.…”

Section: Other Related Workmentioning

confidence: 99%

Approximate Minimum Selection with Unreliable Comparisons

Leucci

¹

,

Liu

²

2021

View full text Add to dashboard Cite

We consider the approximate minimum selection problem in presence of independent random comparison faults. This problem asks to select one of the smallest k elements in a linearly-ordered collection of n elements by only performing unreliable pairwise comparisons: whenever two elements are compared, there is a small probability that the wrong comparison outcome is observed. We design a randomized algorithm that solves this problem with a success probability of at least $$1-q$$ 1 - q for $$q \in (0, \frac{n-k}{n})$$ q ∈ ( 0 , n - k n ) and any $$k \in [1, n-1]$$ k ∈ [ 1 , n - 1 ] using $$O\big ( \frac{n}{k} \big \lceil \log \frac{1}{q} \big \rceil \big )$$ O ( n k ⌈ log 1 q ⌉ ) comparisons in expectation (if $$k \ge n$$ k ≥ n or $$q \ge \frac{n-k}{n}$$ q ≥ n - k n the problem becomes trivial). Then, we prove that the expected number of comparisons needed by any algorithm that succeeds with probability at least $$1-q$$ 1 - q must be $${\varOmega }(\frac{n}{k}\log \frac{1}{q})$$ Ω ( n k log 1 q ) whenever q is bounded away from $$\frac{n-k}{n}$$ n - k n , thus implying that the expected number of comparisons performed by our algorithm is asymptotically optimal in this range. Moreover, we show that the approximate minimum selection problem can be solved using $$O( (\frac{n}{k} + \log \log \frac{1}{q}) \log \frac{1}{q})$$ O ( ( n k + log log 1 q ) log 1 q ) comparisons in the worst case, which is optimal when q is bounded away from $$\frac{n-k}{n}$$ n - k n and $$k = O\big ( \frac{n}{\log \log \frac{1}{q}}\big )$$ k = O ( n log log 1 q ) .

show abstract

“…Several works also considered the top-k recovery problem, which is a weaker form than the full recovery problem considered in this work. Chen et al [2017] give an efficient algorithm for the top K ranking algorithm in the SST class. Chen and Suh [2015] gives a spectral MLE algorithm that can achieve exact ranking with high probability as regularity conditions.…”

Section: Related Workmentioning

confidence: 99%

Achievability and Impossibility of Exact Pairwise Ranking

He¹

2021

Preprint

0

View full text Add to dashboard Cite

We consider the problem of recovering the rank of a set of n items based on noisy pairwise comparisons. We assume the SST class as the family of generative models. Our analysis gave sharp information theoretic upper and lower bound for the exact requirement, which matches exactly in the parametric limit. Our tight analysis on the algorithm induced by the moment method gave better constant in Minimax optimal rate than Shah and Wainwright [2017] and contribute to their open problem. The strategy we used in this work to obtain information theoretic bounds is based on combinatorial arguments and is of independent interest.

show abstract

“…Our algorithm is related, except it performs active learning, using a probability model to decide when to truncate the accumulation of Borda counts and discard or promote an item, while offering precise guarantees on the correctness probability of the top-K set. Chen et al studied bounds for the amount of comparison input that is needed, under the setting in which comparisons can be requested uniformly for all items [10]. In contrast, our algorithm is an on-line one, where comparison requests are dynamically generated for the most needed items.…”

Section: Related Workmentioning

confidence: 99%

Online Top-K Selection in Crowdsourcing Environments

Liang

¹

,

Alfaro

²

2020

Proceedings of 2020 6th International Conference on Computing and Data Engineering

View full text Add to dashboard Cite

Identifying the top-K items in a set of items is a problem that finds applications in many areas, such as recommender systems, social review platforms, online contests, web search, and more. Crowdsourcing provides an effective way to collect input for such tasks with low costs and has attracted significant attention. We consider top-K problems in which the focus consists in selecting the set of top-K elements, regardless of their internal ordering. Past algorithms for top-K problems were generally based on a global sorting, which perform unnecessary work sorting elements that are all selected or all rejected. The exceptions are specialized crowdsourcing algorithms that have been proposed especially for top-K selection; these algorithms, however, require a fixed amount of work to be performed, and produce no useful intermediate answer. We propose here a dynamic top-K selection algorithm that uses crowdsourced comparisons to progressively classify items into selected in top-K, and rejected. As the comparisons proceed, more and more elements are accepted; intermediate results can be provided at any time by returning the already-accepted items along with the best among the ones that are still unclassified. We show that the algorithm we develop is efficient and robust with respect to comparison noise. We illustrate the performance of algorithm both analytically and experimentally, and show that our algorithm can achieve comparable precision in the selection of top-K items with less crowdsourcing work than previous algorithms.

show abstract

Competitive analysis of the top-K ranking problem

Cited by 19 publications

References 22 publications

Approximate Minimum Selection with Unreliable Comparisons

Approximate Minimum Selection with Unreliable Comparisons

Achievability and Impossibility of Exact Pairwise Ranking

Online Top-K Selection in Crowdsourcing Environments

Contact Info

Product

Resources

About