Monitoring of Domain-Related Problems in Distributed Data Streams

Approximation and Online Algorithms

Malatyali

et al. 2018

Self Cite

We consider the scenario of n sensor nodes observing streams of data. The nodes are connected to a central server whose task it is to compute some function over all data items observed by the nodes. In our case, there exists a total order on the data items observed by the nodes. Our goal is to compute the k currently lowest observed values or a value with rank in [(1 − ε)k, (1 + ε)k] with probability (1 − δ). We propose solutions for these problems in an extension of the distributed monitoring model where the server can send broadcast messages to all nodes for unit cost. We want to minimize communication over multiple time steps where there are m updates to a node's value in between queries. The result is composed of two main parts, which each may be of independent interest: * This work was partially supported by the German Research Foundation (DFG) within the Priority Program "Algorithms for Big Data" (SPP 1736).Consider a distributed sensor network which is a system consisting of a huge amount of nodes. Each node continuously observes its environment and measures information (e.g. temperature, pollution or similar parameters). We are interested in aggregations describing the current observations at a central server.To keep the server's information up to date, the server and the nodes can communicate with each other. In sensor networks, however, the amount of such communication is particularly crucial, as communication has the largest impact to energy consumption, which is limited due to battery capacities [11]. Therefore, algorithms aim at minimizing the (total) communication required for computing the respective aggregation function at the server.We consider several ideas to potentially lower the communication used. Each single computation of an aggregate should use as little communication as possible. Computations of the same aggregate should reuse parts of previous computations. Only compute aggregates, if necessary. Recall that the continuous monitoring model creates a new output as often as possible.

Section: Modelmentioning

confidence: 99%

A Communication-Efficient Distributed Data Structure for Top-k and k-Select Queries

Biermeier

Approximation and Online Algorithms

Malatyali

et al. 2018

Self Cite

“…The distributed monitoring model introduced by Cormode, Muthukrishnan, and Yi in [8] allows exchanging single cast messages. The extension we use is the Distributed Monitoring Model with a Broadcast Channel (DMBC-Model) (proposed in [7] and exploited in [3,4,9,10]) which allows, in addition, to broadcast messages from the server to all nodes. Both types of communication are instantaneous and have unit cost per message.…”

Section: The Distributed Monitoring Model With Broadcast Channel (Dmbmentioning

confidence: 99%

A Dynamic Distributed Data Structure for Top-k and k-Select Queries

Adventures Between Lower Bounds and Higher Altitudes

Malatyali

Heide

2018

Self Cite

We consider a scenario where n sensor nodes observe streams of data. The nodes are connected to a central server whose task it is to compute some function over all data items observed by the nodes. Extending the capabilities of the distributed monitoring model from [8], we allow, in addition to sending messages from the sensor nodes to the server, also broadcasts from the server to the sensor nodes (see for example [9]). In this paper, we address the problem of answering Top-k queries (report the k largest data items currently observed) and approximate k-Select queries (report an element with rank close to k). We present a communication-efficient dynamic data structure that supports these queries under updates of the data items arriving at the sensor nodes.

Distributed Data Streams

Castenow

Lecture Notes in Computer Science

Hanselle

et al. 2022

We consider a scenario where a server is wirelessly connected to countless sensor nodes that continuously measure data. The task of the server is to monitor the sensors’ data. More precisely, at each time step the server calculates a function defined over the current measurements of the sensors. Since the sensors only have small computational power and tight battery constraints, the communication between the server and the sensors should be as small as possible, i.e., we aim at minimizing the total number of messages that is transferred.There are various conceivable problems for the setting above. We demonstrate our approaches on the following three: In the Top-k-Value Monitoring Problem, the server aims at identifying the k largest values. The Top-k-Position Monitoring Problem shifts the task to identify the sensors observing these values. Finally, the Count Distinct Monitoring Problem obliges the server to determine the number of distinct values currently observed.For all three problems, we not only present communication-efficient protocols for one time step, we also show how it can be exploited if the input at sensors is similar between consecutive time steps to reduce the total communication on the long term. Thereby, we utilize different techniques – involving sampling, dynamic data structures, filter-based approaches, and combinations of them – to demonstrate their potential and their limits in the broad setting described above.