TKAP: Efficiently processing top-k query on massive data by adaptive pruning

Abstract

In many applications, top-k query is an important operation to return a set of interesting points in a potentially huge data space. The existing algorithms, either maintaining too many candidates, or requiring assistant structures built on the specific attribute subset, or returning results with probabilistic guarantee, cannot process top-k query on massive data efficiently. This paper proposes a sorted-list-based TKAP algorithm, which utilizes some data structures of low space overhead, to efficiently compute top-k results on massive data. In round-robin retrieval on sorted lists, TKAP performs adaptive pruning operation and maintains the required candidates until the stop condition is satisfied. The adaptive pruning operation can be adjusted by the information obtained in round-robin retrieval to achieve a better pruning effect. The adaptive pruning rule is developed in this paper, along with its theoretical analysis. The extensive experimental results, conducted on synthetic and real-life data sets, show the significant advantage of TKAP over the existing algorithms.

Accurate and efficient profile matching in knowledge bases

“…Top-k queries have attracted a lot of attention in connection with ranking problems. Usually, they are investigated in the context of relational databases [27], and the predominant problem associated with them is efficiency [28]. This is particularly the case for join queries [29] or for the determination of a dominant query [30].…”