Scalable delivery of stream query result

Abstract

Continuous queries over data streams typically produce large volumes of result streams. To scale up the system, one should carefully study the problem of delivering the result streams to the end users, which, unfortunately, is often overlooked in existing systems. In this paper, we leverage Distributed Publish/Subscribe System (DPSS), a scalable data dissemination infrastructure, for efficient stream query result delivery. To take advantage of DPSS's multicast-like data dissemination architecture, one has to exploit the common contents among different result streams and maximize the sharing of their delivery. Hence, we propose to merge the user queries into a few representative queries whose results subsume those of the original ones, and disseminate the result streams of these representative queries through the DPSS. To realize this approach, we study the stream query containment theories and propose efficient query grouping and merging algorithms. The proposed approach is nonintrusive and hence can be easily implemented as a middleware to be incorporated into existing stream processing systems. A prototype is developed on top of an open-source stream processing system and results of an extensive performance study on real datasets verify the efficacy of the proposed techniques.

Query Racing: Fast Completeness Certification of Query Results

“…This result is also achieved in other papers [14,20,4,6,16,17] but with different trade offs. We now briefly compare our work with each of them.…”

“…In [17] a scalable technique for verification of queries, in particular for the equi-join operation, is presented. That approach is based on Bloom filters [18].…”

“…Pang et al [4,17] require to sign each tuple and/or to compute complicated hashes which may be a burden for client and users.…”

Processing and Notifying Range Top-k Subscriptions

“…Recent work seek to extend them to support more complex subscriptions (e.g., [8], [11]), or use them for scalable implementation of distributed stream processing [28] and query result caching [13]. The work most relevant to this paper is [8], which discusses scalable processing and dissemination of range top-1 subscriptions.…”

Improving Conjunctive Query Routing in Unstructured P2P Networks

“…As two similar queries can share processing, two similar term sets can share the same partition. This problem is common in traditional publishsubscribe systems (e.g., [18]) and exists in more modern stream-based systems (e.g., [21]). Our work is a special case of query containment in that it only considers exact (term) matching.…”