A novel probabilistic pruning approach to speed up similarity queries in uncertain databases

Abstract

In this paper, we propose a novel, effective and efficient probabilistic pruning criterion for probabilistic similarity queries on uncertain data. Our approach supports a general uncertainty model using continuous probabilistic density functions to describe the (possibly correlated) uncertain attributes of objects. In a nutshell, the problem to be solved is to compute the PDF of the random variable denoted by the probabilistic domination count: Given an uncertain database object B, an uncertain reference object R and a set D of uncertain database objects in a multi-dimensional space, the probabilistic domination count denotes the number of uncertain objects in D that are closer to R than B. This domination count can be used to answer a wide range of probabilistic similarity queries. Specifically, we propose a novel geometric pruning filter and introduce an iterative filter-refinement strategy for conservatively and progressively estimating the probabilistic domination count in an efficient way while keeping correctness according to the possible world semantics. In an experimental evaluation, we show that our proposed technique allows to acquire tight probability bounds for the probabilistic domination count quickly, even for large uncertain databases. I. INTRODUCTIONIn the past two decades, there has been a great deal of interest in developing efficient and effective methods for similarity queries, e.g. k-nearest neighbor search, reverse k-nearest neighbor search and ranking in spatial, temporal, multimedia and sensor databases. Many applications dealing with such data have to cope with uncertain or imprecise data.In this work, we introduce a novel scalable pruning approach to identify candidates for a class of probabilistic similarity queries. Generally spoken, probabilistic similarity queries compute for each database object o ∈ D the probability that a given query predicate is fulfilled. Our approach addresses probabilistic similarity queries where the query predicate is based on object (distance) relations, i.e. the event that an object B belongs to the result set depends on the relation of its distance to the query object R and the distance of another object A to the query object. Exemplarily, we apply our novel pruning method to the most prominent queries of the above mentioned class, including the probabilistic k-nearest neighbor (PkNN) query, the probabilistic reverse k-nearest neighbor (PRkNN) query and the probabilistic inverse ranking query.

Continuous Inverse Ranking Queries in Uncertain Streams

“…The idea is to derive for each database object O, a lower and an upper bound of the probability that O has a higher score than Q. Using these approximations, we can apply the concept of uncertain generating functions [19] in order to obtain an (initial) approximated result of a PIR query, which guarantees that the true result is bounded correctly. The problem at hand is to update these uncertain generating functions efficiently when an update is fetched from the stream.…”

Efficient probabilistic reverse nearest neighbor query processing on uncertain data

“…As shown in [5], the computational complexity is linear in k, yielding a total of O(|DB| 2 × k) for the probabilistic pruning. Verification: The verification step can be performed analogously to the k = 1 case using the algorithm proposed in [6], which has been designed for k ≥ 1.…”

“…In the first experiment (cf. Figure 6(a)), we varied the parameter k. It can be observed that the runtime scales slightly worse than linearly, which can be explained by the usage of uncertain generating functions that show a complexity of O(k 2 ) ( [5]). This is notable, since naive approaches need to consider all N k possible results.…”

“…Analogous to the k = 1 case, we can derive the following bounds for the probability P (A ≺Q B) that A ∈ D \ B is closer to B than Q: a lower bound PLB(A ≺Q B) (using Lemma 2) and an upper bound PUB(A ≺Q B) (using Lemma 3). Given these bounds, we can apply the concept of uncertain generating functions [5] in order to compute for each 0 ≤ j < k a lower bound PLB(#P runers = j) and an upper bound PUB(#P runers = j) of the probability of the random event that for exactly j uncertain objects Ai, Ai ≺Q B is true. A summary of the uncertain generating functions technique is given in Appendix C. Bounds for the probability of the event RkN NQ(Ui) that Ui is a RkNN of Q can then be derived as follows:…”

Voronoi-based nearest neighbor search for multi-dimensional uncertain databases

“…Since expensive integration operations are involved in this step, a number of efficient methods have been proposed. In [11], [36], efficient methods were proposed to generate answer objects' probability bounds without performing expensive integration operations.…”