A Deep Q-Learning Approach for Dynamic Management of Heterogeneous Processors

Thermal cycling as well as spatial and thermal gradient affects the lifetime reliability and performance of heterogeneous multiprocessor systems-on-chips (MPSoCs). Conventional temperature management techniques are not intelligent enough to cater for performance, energy efficiency as well as operating temperature of the system. In this paper we propose a lightweight novel thermal management mechanism (P-EdgeCoolingMode) in the form of intelligent software agent, which monitors and regulates the operating temperature of the CPU cores to improve reliability of the system while catering for performance requirements. P-EdgeCoolingMode is capable of pro-actively monitoring performance and based on the user's demand the agent takes necessary action, making the proposed methodology highly suitable for implementation on existing as well as conceptual Edge devices utilizing heterogeneous MPSoCs with dynamic voltage and frequency scaling (DVFS) capabilities. We validated our methodology on the Odroid-XU4 MPSoC and Huawei P20 Lite (HiSilicon Kirin 659 MPSoC). P-EdgeCoolingMode has been successful to reduce the operating temperature while improving performance and reducing power consumption for chosen test cases than the state-of-the-art. For applications with demanding performance requirement P-EdgeCoolingMode has been found to improve the power consumption by 30.62% at the most in comparison to existing state-of-the-art power management methodologies.

show abstract

“…We ran Streamcluster from PARSEC five times each for different temperature threshold values (

89 °

90 °

91 °

92 °

93 °

94 °

, and

95 °

) and for MRPI [9], Linux governor in performance mode, Linux governor in on‐demand mode and Deep Q‐Learning Dynamic Management [29]. Fig.…”

Section: Resultsmentioning

confidence: 99%

“…In a different study [29], the authors propose a deep Q-learning methodology for dynamic thermal and power management. This approach takes less memory due to the reduced number of bins and hence reducing the number of rows in the Q-table.…”

Section: Related Workmentioning

confidence: 99%

P‐EdgeCoolingMode: an agent‐based performance aware thermal management unit for DVFS enabled heterogeneous MPSoCs

Dey

Singh

McDonald-Maier

2019

IET Computers & Digital Techniques

View full text Add to dashboard Cite

show abstract

“…Its major purpose is to accelerate the running speed of the algorithm. On this basis, the artificial potential field method is added (Gupta et al, 2019 ). The gravitational field is calculated as follows:

Where: k is the gain coefficient, X is the current position of the mobile robot, Xg is the target position, j is the planning adjustment reward, and the relationship between reward and gravity is as follows:

…”

Section: Methodsmentioning

confidence: 99%

The Path Planning of Mobile Robot by Neural Networks and Hierarchical Reinforcement Learning

Liao

2020

Front. Neurorobot.

View full text Add to dashboard Cite

Existing mobile robots cannot complete some functions. To solve these problems, which include autonomous learning in path planning, the slow convergence of path planning, and planned paths that are not smooth, it is possible to utilize neural networks to enable to the robot to perceive the environment and perform feature extraction, which enables them to have a fitness of environment to state action function. By mapping the current state of these actions through Hierarchical Reinforcement Learning (HRL), the needs of mobile robots are met. It is possible to construct a path planning model for mobile robots based on neural networks and HRL. In this article, the proposed algorithm is compared with different algorithms in path planning. It underwent a performance evaluation to obtain an optimal learning algorithm system. The optimal algorithm system was tested in different environments and scenarios to obtain optimal learning conditions, thereby verifying the effectiveness of the proposed algorithm. Deep Deterministic Policy Gradient (DDPG), a path planning algorithm for mobile robots based on neural networks and hierarchical reinforcement learning, performed better in all aspects than other algorithms. Specifically, when compared with Double Deep Q-Learning (DDQN), DDPG has a shorter path planning time and a reduced number of path steps. When introducing an influence value, this algorithm shortens the convergence time by 91% compared with the Q-learning algorithm and improves the smoothness of the planned path by 79%. The algorithm has a good generalization effect in different scenarios. These results have significance for research on guiding, the precise positioning, and path planning of mobile robots.

show abstract

“…Encapsulation of applications' characteristics as a metric called bias (which is defined in Section3) enables making a relevant choice of resource configuration for minimizing energy consumption while respecting performance target [10], [12], [18]. Existing resource management approaches use offline profiling to characterize the applications at design time and use this information at run-time to avoid exhaustive search [10], [16], [21].…”

Section: Introductionmentioning

confidence: 99%

“…Other approaches use on-line prediction strategies, their efficiency and accuracy depend on the amount of power-performance statistics collected over a significant period of execution time [1], [18]. Both off-line and on-line approaches are confined to extract an optimal resource configuration of a single application running in isolation [3], [10], [12], [18], and are thus not readily adaptable to multiple concurrent application scenarios. While some approaches [21], [22] consider multiple applications, those i) overlook the combination of all the existing knobs (DoP, DVFS, and core selection) in heterogeneous platforms [4], [21], [22], [26], ii) ignore dynamic workload scenarios where applications arrive and leave the system in an unknown manner, limiting their efficiency and adaptability in optimizing resource allocation, and iii) do not consider the weight of total energy consumption per application, restricting those from prioritizing among applications [1], [22], [26].…”

Section: Introductionmentioning

confidence: 99%

Concurrent Application Bias Scheduling for Energy Efficiency of Heterogeneous Multi-Core Platforms

Shamsa

Kanduri

Liljeberg

et al. 2022

IEEE Trans. Comput.

View full text Add to dashboard Cite

Minimizing energy consumption of concurrent applications on heterogeneous multi-core platforms is challenging given the diversity in energy-performance profiles of both the applications and hardware. Adaptive learning techniques made the exhaustive Pareto-optimal space exploration practically feasible to identify an energy efficient configuration. Existing approaches consider a single application's characteristic for optimizing energy consumption. However, an optimal configuration for a given application in isolation may not be optimal when other applications are run concurrently. Approaches that consider concurrent application scenarios overlook the weight of total energy consumption per application, restricting them from prioritizing among applications. We address this limitation by considering the mutual effect of concurrent applications on system wide energy consumption to adapt resource configuration at run-time. We characterize each application's power-performance profile as a weighted bias through off-line profiling. We infer this model combined with an on-line predictive strategy to make resource allocation decisions for minimizing energy consumption while honoring performance requirements. The proposed strategy is implemented as a user-space process and evaluated on a heterogeneous hardware platform of Odroid XU3 over the Rodinia benchmark suite. Experimental results show up to 61% of energy saving compared to the standard baseline of Linux governors and up to 27% of energy gain compared to state-of-the-art adaptive learning-based resource management techniques.

show abstract

A Deep Q-Learning Approach for Dynamic Management of Heterogeneous Processors

Cited by 44 publications

References 12 publications

P‐EdgeCoolingMode: an agent‐based performance aware thermal management unit for DVFS enabled heterogeneous MPSoCs

P‐EdgeCoolingMode: an agent‐based performance aware thermal management unit for DVFS enabled heterogeneous MPSoCs

The Path Planning of Mobile Robot by Neural Networks and Hierarchical Reinforcement Learning

Concurrent Application Bias Scheduling for Energy Efficiency of Heterogeneous Multi-Core Platforms

Contact Info

Product

Resources

About