Efficient Scheduling of Streaming Operators for IoT Edge Analytics

Abstract

Data stream processing and analytics (DSPA) applications are widely used to process the ever increasing amounts of data streams produced by highly geographical distributed data sources such as fixed and mobile IoT devices in order to extract valuable information in a timely manner for real-time actuation. To efficiently handle this ever increasing amount of data streams, the emerging Edge/Fog computing paradigms is used as the middle-tier between the Cloud and the IoT devices to process data streams closer to their sources and to reduce the network resource usage and network delay to reach the Cloud. In this paper, we account for the fact that both network resources and computational resources can be limited and shareable among multiple DSPA applications in the Edge-Fog-Cloud architecture, hence it is necessary to ensure their efficient usage. In this respect, we propose a resource-aware and time-efficient heuristic called SOO that identifies a good DSPA operator placement on the Edge-Fog-Cloud architecture towards optimizing the trade-off between the computational and network resource usage. Via thorough simulation experiments, we show that the solution provided by SOO is very close to the optimal one while the execution time is considerably reduced.

Scheduling Continuous Operators for IoT edge Analytics with Time Constraints

“…To capture the fact that computational resources allocated to a DSPA application at different layers of an Edge-Fog-Cloud architecture are limited, we weight the computational resource usage of different operators sharing a node by the inverse of the available computational resources at this node. Our model relies on the following intuitive assumptions [11], [17]: (i) the computational resources are practically unlimited in the Cloud, cmC → ∞, thanks to the on-demand resource scaling, and hence the weight in the cloud is practically zero, 1 cm C → 0; (ii) the computational resources of Fog and Edge nodes are limited as they can not be scaled on demand, and thus the weight in a Fog (or Edge) node is non-zero,…”

“…Unlike [11], we consider the fact that the network delays and network bandwidths of each individual WAN links can be dynamic with regard to the network conditions [5]. Thus, we need to differentiate network links not only according to their delay [19] but also by their available bandwidth [11].…”

“…Unlike [11], we consider the fact that the network delays and network bandwidths of each individual WAN links can be dynamic with regard to the network conditions [5]. Thus, we need to differentiate network links not only according to their delay [19] but also by their available bandwidth [11]. In this respect, the bandwidth usage of a network link is weighted by the inverse of its available bandwidth multiplied by the ratio between its network delay and the maximum network delay (ndmax) among all links in the hierarchical WAN of resources: ndmax = max i,j…”

“…the evolution of the workload [10]. In our previous work [11], we introduced a cost model of both network (bandwidth, delay) and computational resources (CPU/RAM) of network nodes. Using this model, we formulated the problem of scheduling DSPA operators between the Fog and the Cloud nodes to process data streams produced by mobile IoT devices at the Edge as a single objective optimization (SOO) problem of the combined resource cost subject to the constraints of the computational resource usage, cloud bandwidth usage and operator replicability constraint.…”

“…In this section we present the core model of Edge-Fog-Cloud architectures and DSPA applications and we formulate the resource allocation problem (for more details, readers are referred to [11]).…”