Reinforcement Learning-Based Inter- and Intra-Application Thermal Optimization for Lifetime Improvement of Multicore Systems

Abstract

The thermal profile of multicore systems vary both within an application's execution (intra) and also when the system switches from one application to another (inter). In this paper, we propose an adaptive thermal management approach to improve the lifetime reliability of multicore systems by considering both inter-and intra-application thermal variations. Fundamental to this approach is a reinforcement learning algorithm, which learns the relationship between the mapping of threads to cores, the frequency of a core and its temperature (sampled from on-board thermal sensors). Action is provided by overriding the operating system's mapping decisions using affinity masks and dynamically changing CPU frequency using in-kernel governors. Lifetime improvement is achieved by controlling not only the peak and average temperatures but also thermal cycling, which is an emerging wear-out concern in modern systems. The proposed approach is validated experimentally using an Intel quad-core platform executing a diverse set of multimedia benchmarks. Results demonstrate that the proposed approach minimizes average temperature, peak temperature and thermal cycling, improving the mean-timeto-failure (MTTF) by an average of 2x for intra-application and 3x for inter-application scenarios when compared to existing thermal management techniques. Furthermore, the dynamic and static energy consumption are also reduced by an average 10% and 11% respectively.

System-Level Analysis and Design under Uncertainty

“…In [27], a design methodology that minimizes energy consumption of and temperature-induced wear on multiprocessor systems is introduced; yet neither energy nor temperature is modeled with an awareness of uncertainty due to process variation. A similar observation can be made with respect to the work reported in [28] where a reinforcementlearning algorithm is used in order to improve the lifetime of multiprocessor systems. An extensive survey of reliability-aware system-level design techniques given in [26] confirms the trend emphasized above: the widespread device-level models of failure mechanisms generally ignore the impact of process variation on temperature.…”

“…Similarly, workload uncertainty has not been deprived of attention; see, for instance, [32,88,96,98,105,124]. Aging uncertainty has also been studied extensively in the literature; see, for instance, [24,28,39,50,61,83]. However, certain important problems have not been addressed yet, and in the case of the ones that have been considered, the proposed solutions are often restricted in use, which is due in part to the unrealistic assumptions that these solutions make.…”

Adaptive and Hierarchical Runtime Manager for Energy-Aware Thermal Management of Embedded Systems

“…However, as shown in [Faruque et al 2010], these approaches cannot guarantee to minimize a system's thermal overhead effectively for all applications. A cross-layer thermal optimization technique is proposed in [Das et al 2014] to manage temperature-related emergencies. Although these studies have shown improvement in thermal profile leading to extended lifetime reliability using scaled voltage and frequency, thermal cycling and energy consumption are not jointly addressed.…”

“…As shown in [Das et al 2014], temperature of an embedded system can be controlled significantly by controlling the processor power states (i.e., their voltage and frequency) and the application thread allocation (that limits context switching). However, the amount of thermal control achieved using these control levers is dependent on the application, its cross-layer interaction with the system software and the hardware, and also on the working environment.…”

“…As such, static compile-time policies [Rai et al 2011;Schor et al 2013;Das et al 2015a] (with limited knowledge of application-specific variations) are often outperformed, even by naive run-time managers, both in terms of thermal overhead and energy consumption -the two key design aspects of modern systems. This has motivated researchers in recent years to investigate run-time approaches, thriving the development of intelligent run-time systems for energy and thermal management [Cochran et al 2011b;Javaid et al 2011;Juan et al 2013;Ye and Xu 2014;Srinivasan et al 2004;Sharifi et al 2013;Shi et al 2013;Faruque et al 2010;Ge and Qiu 2011;Coskun et al 2009a;Mercati et al 2013;Das et al 2014;Coskun et al 2009b;Ebi et al 2009;Ebi et al 2011;Shen et al 2012].…”