Operator placement for in-network stream query processing

Abstract

In sensor networks, data acquisition frequently takes place at low-capability devices. The acquired data is then transmitted through a hierarchy of nodes having progressively increasing network bandwidth and computational power. We consider the problem of executing queries over these data streams, posed at the root of the hierarchy. To minimize data transmission, it is desirable to perform "in-network" query processing: do some part of the work at intermediate nodes as the data travels to the root. Most previous work on in-network query processing has focused on aggregation and inexpensive filters. In this paper, we address in-network processing for queries involving possibly expensive conjunctive filters, and joins. We consider the problem of placing operators along the nodes of the hierarchy so that the overall cost of computation and data transmission is minimized. We show that the problem is tractable, give an optimal algorithm, and demonstrate that a simpler greedy operator placement algorithm can fail to find the optimal solution. Finally we define a number of interesting variations of the basic operator placement problem and demonstrate their hardness.

Maximum lifetime continuous query processing in wireless sensor networks

“…Srivastava et al [30] consider the problem of placing operators onto a hierarchy of network nodes with progressively increasing computational power and network bandwidth, such that the total cost of computation and communication is minimized. We assume a different network model in which sensors are homogeneous and are energy limited, and a different goal of optimizing the system lifetime, which does not necessarily result from minimizing the total cost of computation and communication.…”

Biologically-Inspired Distributed Middleware Management for Stream Processing Systems

“…Another approach computes placement by minimizing the average time, estimated by a queueing model, required for an event originating at the producer to reach its destined consumer, also using a centralized controller [16]. Operator placement has also been examined in the context of in-network stream query processing for sensor network environments with progressively increasing computational power network bandwidth up a hierarchy of nodes [14]. This approach provides theoretical analysis of a centralized placement algorithm that minimizes the total cost of computation as well as communication, but does not consider how the algorithm will respond to dynamic changes during runtime.…”

“…Obtaining an optimal solution to the static task assignment problem, where the problem characteristics do not change, is computationally intractable [14]. The problem we address in this paper is even more difficult, due to the following additional requirements: -Dynamic and incremental : We cannot assume that the entire flow graph and network description is known once and for all prior to deployment.…”

Efficient Dynamic Operator Placement in a Locally Distributed Continuous Query System

“…[10] studied the static operator placement in a hierarchical stream acquisition architecture, which is much different from our system architecture. Load balancing is also ignored in this piece of work.…”