Class-based continuous query scheduling for data streams

Abstract

Wireless sensor networks link the physical and digital worlds enabling both surveillance as well as scientific exploration. In both cases, on-line detection of interesting events can be accomplished with continuous queries (CQs) in a Data Stream Management System (DSMS). However, the qualityof-service requirements of detecting these events are different for different monitoring applications. The CQs for detecting anomalous events (e.g., fire, flood) have stricter response time requirements over CQs which are for logging and keeping statistical information of physical phenomena. In this work, we are proposing the Continuous Query Class (CQC) scheduler, a new scheduling policy which employs two-level scheduling that is able to handle different ranks of CQ classes. It provides the lowest response times for classes of critical CQs, while at the same time keeping reasonable response times for the other classes down the rank. We have implemented CQC in the AQSIOS prototype DSMS and evaluated it against existing scheduling policies under different workloads.

Avoiding class warfare: managing continuous queries with differentiated classes of service

“…In this implementation, we use a two-level, class-based DSMS scheduler proposed in [10], called CQC. As indicated in Sect.…”

“…The modified HR scheduler aims to preserve the goal of the original priority-based HR scheduler to minimize the average response time, yet eliminates starvation within a class. More details on CQC can be found at [10].…”

“…For example, assuming that semantic shedding is desired for a class of 2 CQs, Q 1 and Q 2 , each of which has input tuples containing integer keys in [1][2][3][4][5][6][7][8][9][10]. For Q 1 , output with keys [9][10] is more important than those in [1][2][3][4][5][6][7][8], while for Q 2 those in [1][2] is more important than the others. When ALoMa determines that, say, 20 % of the current load needs to be shed, the semantic shedder will take that 20 % as input for its algorithm.…”

“…When ALoMa determines that, say, 20 % of the current load needs to be shed, the semantic shedder will take that 20 % as input for its algorithm. Correspondingly, the semantic shedder decides that for Q 1 , it drops 10 8 × 20 % tuples with keys in [1][2][3][4][5][6][7][8], while for Q 2 it drops 10 8 × 20 % tuples with keys in [3][4][5][6][7][8][9][10], keeping the whole important range (assuming a uniform distribution of the keys). Note that this assumes queries which have different semantic on incoming tuples, as in the case of Q 1 and Q 2 in this example, do not share operators with each other.…”

“…Previous works have partially addressed these requirements, either through scheduling (e.g., [10,15]) or through load shedding (e.g., [17,42]), yet these were only considered in isolation. Clearly, enforcing worst-case QoS in overload situations while providing prioritized QoD for query classes requires the participation of both the scheduler and the load manager (i.e., load shedder): The load manager decides how much data to drop from each class, whereas the scheduler decides how much processing time each query has, which consequently governs how much data the class can process in a period.…”