Crowdsourced Top-k Queries by Confidence-Aware Pairwise Judgments

Kou, Ngai Meng; Li, Yan; Wang, Hao; Hou, U Leong; Gong, Zhiguo

doi:10.1145/3035918.3035953

Cited by 23 publications

(22 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The SPR method [6] is most relevant to our scheme TMG. Meanwhile, SPR is also a very classical method for our application scenarios.…”

Section: Methodsmentioning

confidence: 99%

“…The core idea of TMG is to multilayer group the candidate dataset based on k value to almost minimize the top-k query cost. According to k value, we first compute the average score of candidate dataset D as reference item r. Second, we compute best range c based on r according to literature [6]. Third, we group the candidate dataset into three datasets: winner set, middle set and loser set based on best range c. According to k value, we further group the winner set.…”

Section: Top-k Query Schemementioning

confidence: 99%

“…Therefore, this paper involves the following three algorithms: Grouping, Top-k, and Query. Compute best range c by r according to literature [6]…”

Section: Top-k Query Schemementioning

confidence: 99%

“…In the current top-k query methods, SPR [6] has three shortcomings: high sorting costs, insufficient grouping and high memory consumption. The sorting cost of tree selection [7] is also high.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

An Efficient Top-k Query Scheme Based on Multilayer Grouping

et al. 2019

View full text Add to dashboard Cite

The top-k query is to find the k data that has the highest scores from a candidate dataset. Sorting is a common method to find out top-k results. However, most of existing methods are not efficient enough. To remove this issue, we propose an efficient top-k query scheme based on multilayer grouping. First, we find the reference item by computing the average score of the candidate dataset. Second, we group the candidate dataset into three datasets: winner set, middle set and loser set based on the reference item. Third, we further group the winner set to the second-layer three datasets according to k value. And so on, until the data number of winner set is close to k value. Meanwhile, if k value is larger than the data number of winner set, we directly return the winner set to the user as a part of top-k results almost without sorting. In this case, we also return the top results with the highest scores from the middle set almost without sorting. Based on above innovations, we almost minimize the sorting. Experimental results show that our scheme significantly outperforms the current classical method on the performance of memory consumption and top-k query.

show abstract

“…The SPR method [6] is most relevant to our scheme TMG. Meanwhile, SPR is also a very classical method for our application scenarios.…”

Section: Methodsmentioning

confidence: 99%

Section: Top-k Query Schemementioning

confidence: 99%

“…Therefore, this paper involves the following three algorithms: Grouping, Top-k, and Query. Compute best range c by r according to literature [6]…”

Section: Top-k Query Schemementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An Efficient Top-k Query Scheme Based on Multilayer Grouping

et al. 2019

View full text Add to dashboard Cite

show abstract

“…For instance, picking the younger photo between a child and an adult should need fewer judgments than pick between two adolescents. Based on this observation, our prior work [7] studied a crowdsourced confidence-aware query processing which takes the judgment difficulty (estimated by the confidence level in statistics) into the query processing. As shown in [7], the confidence-aware solution not only secures the query answering quality but also minimizes the crowdsourcing cost.…”

Section: Introductionmentioning

confidence: 99%

A confidence-aware top-k query processing toolkit on crowdsourcing

Kou

Wang

et al. 2017

Proc. VLDB Endow.

Self Cite

View full text Add to dashboard Cite

Ranking techniques have been widely used in ubiquitous applications like recommendation, information retrieval, etc. For ranking computation hostile but human friendly items, crowdsourcing is considered as an emerging technique to process the ranking by human power. However, there is a lack of an easy-to-use toolkit for answering crowdsourced top-k query with minimal effort. In this work, we demonstrate an interactive programming toolkit that is a unified solution for answering the crowdsourced top-k queries. The toolkit employs a new confidenceaware crowdsourced top-k algorithm, SPR. The whole progress of the algorithm is monitored and visualized to end users in a timely manner. Besides the visualized result and the statistics, the system also reports the estimation of the monetary cost and the breakdown of each phase. Based on the estimation, end users can strike a balance between the budget and the quality through the interface of this toolkit.

show abstract

A General Early-Stopping Module for Crowdsourced Ranking

Shan

Mamoulis

et al. 2020

Database Systems for Advanced Applications

View full text Add to dashboard Cite

Crowdsourced Top-k Queries by Confidence-Aware Pairwise Judgments

Cited by 23 publications

References 27 publications

An Efficient Top-k Query Scheme Based on Multilayer Grouping

An Efficient Top-k Query Scheme Based on Multilayer Grouping

A confidence-aware top-k query processing toolkit on crowdsourcing

A General Early-Stopping Module for Crowdsourced Ranking

Contact Info

Product

Resources

About