Scheduling Continuous Operators for IoT edge Analytics with Time Constraints

Abstract

Data stream processing and analytics (DSPA) engines are used to extract in (near) real-time valuable information from multiple IoT data streams. Deploying DSPA applications at the IoT network edge through Edge/Fog architectures is currently one of the core challenges for reducing both network delays and network bandwidth usage to reach the Cloud. In this paper, we address the problem of scheduling continuous DSPA operators to Fog-Cloud nodes featuring both computational and network resources. We are paying particular attention to the dynamic workload of these nodes due to variability of IoT data stream rates and the sharing of nodes' resources by multiple DSPA applications. In this respect, we propose TSOO, a resource-aware and time-efficient heuristic algorithm that takes into account the limited Fog computational resources, the real-time response constraints of DSPA applications, as well as, congestion and delay issues on Fog-to-Cloud network resources. Via extensive simulation experiments, we show that TSOO approximates an optimal operators' placement with a low execution cost.Index Terms-IoT (Edge), data stream, Fog, Cloud, continuous operator, scheduling, real time, queuing model • we define the response time of a DSPA application by abstracting each operator by a queuing model [12];

Adaptive Scheduling of Continuous Operators for IoT Edge Analytics

“…To objectively evaluates the cost of request to use a resource, we need also to take into account the state of the resource if it is selected to be used. Therefore, unlike in [37,10,38], we introduce a new weighting of the usage of the EFC resources based on which we assess the cost of using this resource. Table 4 summarises the parameters you use to model the resource usage cost.…”

“…In our previous work [10], we address the problem of scheduling a DSPA application on dedicated fog and cloud resources with the objective of simultaneously minimising the computational resource capacities of the fog nodes and the network resource capacities of using the WAN links to reach the cloud, while satisfying the application response time constraint. A static heuristic algorithm was proposed: it required the scheduling solution to be computed from scratch each time.…”

“…• Simulation experiments: The proposed aTSOO-H algorithm is evaluated through extensive simulations based on real-world data sets. When compared with the baseline solution [10] and the solutions from related work [8], the results show that aTSOO-H achieves a high dynamic deployment success rate of DSPA applications, i.e., the best trade-off between resource usage cost, response time satisfaction and lower rescheduling cost. Rescheduling cost is defined as the difference between the number of operators that a DSPA application contains before after its scheduling.…”

A Network Design Scheme in Delay Sensitive Monitoring Services

“…Several types of research works have been studied on delay reduction in IoT services using multiple clouds via a wide-area network [12][13][14][15][16][17][18]. These works are based on the distributed allocation of functions at multiple clouds and focus on controlling the transmitting and processing delays.…”

“…In distributed processing, a method for satisfying delay constraints by managing tradeoffs between delay and resources of computation and network was reported in [16]. Distributed processing systems that offload processing to reduce delay face the issue of not being able to process heavy tasks due to a lack of available resources or limited computing power.…”