The Internet of Things Resource Management Challenge

Kliem, Andreas; Kao, Odej

doi:10.1109/dsdis.2015.21

Cited by 19 publications

(14 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…At the same time, IoT data is voluminous and complex for which scalable systems are necessary [34]. CPU processing time, I/O time, storage resources, and energy efficiency are examples of resource constraints that will have an adverse impact on efficient data processing, thus rendering IoT resource management a challenge [35,36].…”

Section: Green Iot and Big Data (Green Software Engineering)mentioning

confidence: 99%

Green Cloud Software Engineering for Big Data Processing

et al. 2020

View full text Add to dashboard Cite

Internet of Things (IoT) coupled with big data analytics is emerging as the core of smart and sustainable systems which bolsters economic, environmental and social sustainability. Cloud-based data centers provide high performance computing power to analyze voluminous IoT data to provide invaluable insights to support decision making. However, multifarious servers in data centers appear to be the black hole of superfluous energy consumption that contributes to 23% of the global carbon dioxide (CO2) emissions in ICT (Information and Communication Technology) industry. IoT-related energy research focuses on low-power sensors and enhanced machine-to-machine communication performance. To date, cloud-based data centers still face energy–related challenges which are detrimental to the environment. Virtual machine (VM) consolidation is a well-known approach to affect energy-efficient cloud infrastructures. Although several research works demonstrate positive results for VM consolidation in simulated environments, there is a gap for investigations on real, physical cloud infrastructure for big data workloads. This research work addresses the gap of conducting real physical cloud infrastructure-based experiments. The primary goal of setting up a real physical cloud infrastructure is for the evaluation of dynamic VM consolidation approaches which include integrated algorithms from existing relevant research. An open source VM consolidation framework, Openstack NEAT is adopted and experiments are conducted on a Multi-node Openstack Cloud with Apache Spark as the big data platform. Open sourced Openstack has been deployed because it enables rapid innovation, and boosts scalability as well as resource utilization. Additionally, this research work investigates the performance based on service level agreement (SLA) metrics and energy usage of compute hosts. Relevant results concerning the best performing combination of algorithms are presented and discussed.

show abstract

Section: Green Iot and Big Data (Green Software Engineering)mentioning

confidence: 99%

Green Cloud Software Engineering for Big Data Processing

et al. 2020

View full text Add to dashboard Cite

show abstract

“…These management capabilities are hosted by an independent Domain Operator not participating in the resource sharing processes (similar to the DNS system). However, resource sharing is not restricted to the set of devices contained in one Domain.As described by the general concept of sharing devices for the purpose of increasing resource utilization [12], interactions crossing Domain boundaries are possible. Therefore, the Domain Operators need to negotiate access to so called remote resources on behalf of the users be part of their Domain.…”

Section: Device Cloud Domain Networkmentioning

confidence: 99%

“…As a result, the utilization of compute and storage capabilities can be increased and the communication demand and latency can be reduced, because edge devices are used for data (pre-) processing and aggregation. In addition, a resource oriented view on the available IoT infrastructure enables new ways of mobility, scalability, and upcoming paradigms like community based interaction and sharing of IoT resources [12,14].…”

Section: Introductionmentioning

confidence: 99%

Edge Resource Utilization Using OS-Level Virtualization on Smart Devices

Renner

Meldau

Kliem

2016

JSN

Self Cite

View full text Add to dashboard Cite

Following the Internet of Things (IoT) paradigm, more and more connected and embedded devices like sensors, smart phones, or entertainment devices are becoming available surrounding us. This increasing number of IoT devices leads to an increasing amount of resources made available to the users. For instance, if devices like smart phones or smart TVs are considered, this includes computing and storage resources. In this paper we propose a container-based resource allocation scheme to increase the utilization of these IoT resources and made them shareable between different users due to cloud-centric IoT deployments. The approach allows various applications and users to dynamically allocate resources offered by edge devices and process IoT data close to the source. The approach is evaluated regarding its feasibility in terms of performance on a exemplary resource constrained IoT device.

show abstract

“…First, containerized applications and virtualized cloud infrastructures allow us to deploy and operate multi-tenant applications cost-efficiently through standard, high-level interfaces; these increase the complexity and heterogeneity of cluster workloads and systems in terms of resource management [18]- [21]. Emergent technology trends such as Internet of Things (IoT), edge computing, and network function virtualization (NFV) further increase the complexity by incorporating a new type of devices into distributed cluster environments [22], [23]. We employ attentive embedding for representing such complex properties of cluster entities, and implement a layered attention mechanism for capturing the bi-directional impact of entity pairs within this hierarchy.…”

Section: Introductionmentioning

confidence: 99%

SCARL: Attentive Reinforcement Learning-Based Scheduling in a Multi-Resource Heterogeneous Cluster

et al. 2019

View full text Add to dashboard Cite

Advanced reinforcement learning (RL) technologies have recently increased the opportunity for automating several tasks in cluster management at scale by exploiting repetitive logs of cluster operation and building a learning model for resource allocation and job scheduling. Yet, this trend of adopting RL in the domain of cluster management has not fully addressed the diversity and heterogeneity of jobs and machines in modern cluster environments. In this paper, we present an RL-based scheduler for a multi-resource cluster, namely SCARL (SCheduler with Attentive Reinforcement Learning), concentrating on intricate cluster operating conditions with different resource requirements and capabilities. Specifically, we employ attentive embedding and factored-action scheduling that together efficiently incorporate time-varying interdependency of jobs and machines in RL processing; they enable an end-to-end scalable policy for scheduling diverse jobs on heterogeneous machines. To the best of our knowledge, we are the first to employ attention mechanism in RL-based cluster resource management. Through experiments, we demonstrate that our approach is competitive with existing heuristic methods under various cluster simulation configurations, e.g., an average 9.2 % enhancement in slowdown over the shortest job first algorithm. Additionally, the approach yields stable performance with our test cluster for running synthetic workloads based on real traces. INDEX TERMS Cluster resource management, attentive reinforcement learning, attention, attentive embedding.

show abstract

The Internet of Things Resource Management Challenge

Cited by 19 publications

References 16 publications

Green Cloud Software Engineering for Big Data Processing

Green Cloud Software Engineering for Big Data Processing

Edge Resource Utilization Using OS-Level Virtualization on Smart Devices

SCARL: Attentive Reinforcement Learning-Based Scheduling in a Multi-Resource Heterogeneous Cluster

Contact Info

Product

Resources

About