Reinforcement Learning for Thermal-aware Many-core Task Allocation

Lu, Shiting (Justin); Tessier, Russell; Burleson, Wayne

doi:10.1145/2742060.2742078

Cited by 28 publications

(12 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several works employ RL for temperature optimization. The work in [24] performs migration for temperature minimization based on per-core temperature measurements. In [25], the temperature is minimized via mapping applications at arrival time.…”

Section: Related Workmentioning

confidence: 99%

“…The action space is selected the same as with our IL technique, which is also the same as in [24]. There is one action per core, indicating a migration to this core, i.e., in total 8 actions.…”

Section: State Action and Rewardmentioning

confidence: 99%

“…The reward function needs to combine the objective (temperature minimization) and constraint (QoS target) into a single scalar value. The objective is similar to [24], which only rewards a low temperature T : r=80 • C−T . We extend it to penalize QoS violations:…”

Section: State Action and Rewardmentioning

confidence: 99%

See 2 more Smart Citations

NPU-Accelerated Imitation Learning for Thermal Optimization of QoS-Constrained Heterogeneous Multi-Cores

Rapp¹,

Khdr²,

Krohmer³

et al. 2022

Preprint

View full text Add to dashboard Cite

Application migration and dynamic voltage and frequency scaling (DVFS) are indispensable means for fully exploiting the available potential in thermal optimization of a heterogeneous clustered multi-core processor under user-defined quality of service (QoS) targets. However, selecting the core to execute each application and the voltage/frequency (V/f) levels of each cluster is a complex problem because 1) the diverse characteristics and QoS targets of applications require different optimizations, and 2) per-cluster DVFS requires a global optimization considering all running applications. State-of-the-art resource management techniques for power or temperature minimization either rely on measurements that are often not available (such as power) or fail to consider all the dimensions of the problem (e.g., by using simplified analytical models). Imitation learning (IL) enables to use the optimality of an oracle policy, yet at low run-time overhead, by training a model from oracle demonstrations. We are the first to employ IL for temperature minimization under QoS targets. We tackle the complexity by training a neural network (NN) and accelerate the NN inference using a neural processing unit (NPU). While such NN accelerators are becoming increasingly widespread on end devices, they are so far only used to accelerate user applications. In contrast, we use an existing accelerator on a real platform to accelerate NN-based resource management. Our evaluation on a HiKey 970 board with an Arm big.LITTLE CPU and an NPU shows significant temperature reductions at a negligible run-time overhead, with unseen applications and different cooling than used for training.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: State Action and Rewardmentioning

confidence: 99%

See 1 more Smart Citation

NPU-Accelerated Imitation Learning for Thermal Optimization of QoS-Constrained Heterogeneous Multi-Cores

Rapp¹,

Khdr²,

Krohmer³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In contrast, machine learning-based techniques can observe, learn, and adapt to different working environments, making them a potential choice to be employed in varying conditions and workloads. Machine learning (ML) based predictors such as neural networks (Rong Ye and Qiang Xu, 2012), Bayesian learning (Jung and Pedram, 2010;Yanzhi Wang et al, 2013), reinforcement learning (Hantao et al, 2014;Xu et al, 2014Xu et al, , 2018Lu, Tessier and Burleson, 2015;D. et al, 2016), and regression analysis (Manoj P. D., Yu, and Wang, 2015;Sheng Yang et al, 2015;Sayadi et al, 2017) are also widely utilized for prediction and to perform Dynamic voltage and frequency scaling (DVFS).…”

Section: Bayesian Learning Reinforcement Learning and Regression Anal...mentioning

confidence: 99%

Recent Trends in Artificial Intelligence-inspired Electronic Thermal Management

Chharia¹,

Mehta²,

Gupta³

et al. 2021

Preprint

View full text Add to dashboard Cite

The rise of computation-based methods in thermal management has gained immense attention in recent years due to the ability of deep learning to solve complex 'physics' problems, which are otherwise difficult to be approached using conventional techniques. Thermal management is required in electronic systems to keep them from overheating and burning, enhancing their efficiency and lifespan. For a long time, numerical techniques have been employed to aid in the thermal management of electronics. However, they come with some limitations. To increase the effectiveness of traditional numerical approaches and address the drawbacks faced in conventional approaches, researchers have looked at using artificial intelligence at various stages of the thermal management process. The present study discusses in detail, the current uses of deep learning in the domain of 'electronic' thermal management.

show abstract

“…In addition to energy control, ML offers an approach to allocate resources to tasks or tasks to resources by predicting the impact of various configurations on long-term performance. Lu et al [82] proposed a thermal-aware Q-learning method for many-core task allocation. The agent considered only current temperature (i.e., no application profiling or hardware counters), receiving higher rewards for task assignments resulting in greater thermal headroom.…”

Section: Task Allocation and Resource Managementmentioning

confidence: 99%

A Survey of Machine Learning Applied to Computer Architecture Design

Penney¹,

Chen

2019

Preprint

View full text Add to dashboard Cite

Machine learning has enabled significant benefits in diverse fields, but, with a few exceptions, has had limited impact on computer architecture. Recent work, however, has explored broader applicability for design, optimization, and simulation. Notably, machine learning based strategies often surpass prior state-of-the-art analytical, heuristic, and human-expert approaches. This paper reviews machine learning applied system-wide to simulation and run-time optimization, and in many individual components, including memory systems, branch predictors, networks-on-chip, and GPUs. The paper further analyzes current practice to highlight useful design strategies and identify areas for future work, based on optimized implementation strategies, opportune extensions to existing work, and ambitious long term possibilities. Taken together, these strategies and techniques present a promising future for increasingly automated architectural design.

show abstract

Reinforcement Learning for Thermal-aware Many-core Task Allocation

Cited by 28 publications

References 15 publications

NPU-Accelerated Imitation Learning for Thermal Optimization of QoS-Constrained Heterogeneous Multi-Cores

NPU-Accelerated Imitation Learning for Thermal Optimization of QoS-Constrained Heterogeneous Multi-Cores

Recent Trends in Artificial Intelligence-inspired Electronic Thermal Management

A Survey of Machine Learning Applied to Computer Architecture Design

Contact Info

Product

Resources

About