Deep reinforcement learning towards real-world dynamic thermal management of data centers

Zhang, Qingang; Zeng, Wei; Lin, Qinjie; Chng, Chin-Boon; Chui, Chee-Kong; Lee, Poh Seng

doi:10.1016/j.apenergy.2022.120561

Cited by 12 publications

(6 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep Reinforcement Learning (DRL) shows promise in dynamic thermal management in DCs, specifically for reducing energy consumption via Heating, Ventilation, and Air Conditioning (HVAC) system control (Zhang et al 2023b,a;Mahbod et al 2022;Biemann et al 2021;Wang et al 2022;Naug et al 2023a,b). However, the real-world deployment of DRL-based systems is complicated by their sensitivity to hyperparameters, reward functions, and work scenarios (Zhang et al 2023b;Mahbod et al 2022;Biemann et al 2021;Wang et al 2022). Moreover, ensuring safety and satisfying operational constraints, especially for HVAC system control, is another challenge (Zhang et al 2023b;Wang et al 2022).…”

Section: Related Work Energy Savingsmentioning

confidence: 99%

“…However, the real-world deployment of DRL-based systems is complicated by their sensitivity to hyperparameters, reward functions, and work scenarios (Zhang et al 2023b;Mahbod et al 2022;Biemann et al 2021;Wang et al 2022). Moreover, ensuring safety and satisfying operational constraints, especially for HVAC system control, is another challenge (Zhang et al 2023b;Wang et al 2022).…”

Section: Related Work Energy Savingsmentioning

confidence: 99%

“…Despite the challenges, DRL has shown potential for energy savings in DCs. DRL-based strategies have achieved up to 12% savings compared to default controllers (Zhang et al 2023b), and 8.84% compared to reference controllers. Additional savings of up to 5.5% have been noted in tropical climates (Mahbod et al 2022), while in simulated environments, a reduction of at least 10% in energy consumption has been observed (Biemann et al 2021).…”

Section: Related Work Energy Savingsmentioning

confidence: 99%

See 2 more Smart Citations

Carbon Footprint Reduction for Sustainable Data Centers in Real-Time

Sarkar,

Naug,

Luna

et al. 2024

AAAI

View full text Add to dashboard Cite

As machine learning workloads are significantly increasing energy consumption, sustainable data centers with low carbon emissions are becoming a top priority for governments and corporations worldwide. This requires a paradigm shift in optimizing power consumption in cooling and IT loads, shifting flexible loads based on the availability of renewable energy in the power grid, and leveraging battery storage from the uninterrupted power supply in data centers, using collaborative agents. The complex association between these optimization strategies and their dependencies on variable external factors like weather and the power grid carbon intensity makes this a hard problem. Currently, a real-time controller to optimize all these goals simultaneously in a dynamic real-world setting is lacking. We propose a Data Center Carbon Footprint Reduction (DC-CFR) multi-agent Reinforcement Learning (MARL) framework that optimizes data centers for the multiple objectives of carbon footprint reduction, energy consumption, and energy cost. The results show that the DC-CFR MARL agents effectively resolved the complex interdependencies in optimizing cooling, load shifting, and energy storage in real-time for various locations under real-world dynamic weather and grid carbon intensity conditions. DC-CFR significantly outperformed the industry-standard ASHRAE controller with a considerable reduction in carbon emissions (14.5%), energy usage (14.4%), and energy cost (13.7%) when evaluated over one year across multiple geographical regions.

show abstract

Section: Related Work Energy Savingsmentioning

confidence: 99%

Section: Related Work Energy Savingsmentioning

confidence: 99%

Section: Related Work Energy Savingsmentioning

confidence: 99%

See 1 more Smart Citation

Carbon Footprint Reduction for Sustainable Data Centers in Real-Time

Sarkar,

Naug,

Luna

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Machine learning applications are established as equally powerful tools in the controllers of motor drives [26][27][28][29]. In the work presented in [26], a hysteresis controller behavior was trained offline to an ANN topology to generate the desired switching pa ern.…”

Section: Introduction and Related Workmentioning

confidence: 99%

“…Wishart et al [27] worked on two ANN controllers trained to control the stator current and rotor speed adaptively to capture the induction machine dynamics. Implementations of ANN as speed controllers have been presented in [28,29]. The speed controllers were trained offline using simulation, but the weight and biases were updated online when the difference between the actual output and the targeted output of the ANN controller exceeds a preset value.…”

Section: Introduction and Related Workmentioning

confidence: 99%

Synergizing Transfer Learning and Multi-Agent Systems for Thermal Parametrization in Induction Traction Motors

Mehboob,

Fattouh,

Sahoo

2024

Applied Sciences

View full text Add to dashboard Cite

Maintaining optimal temperatures in the critical parts of an induction traction motor is crucial for railway propulsion systems. A reduced-order lumped-parameter thermal network (LPTN) model enables computably inexpensive, accurate temperature estimation; however, it requires empirically based parameter estimation exercises. The calibration process is typically performed in labs in a controlled experimental setting, which is associated with a lot of supervised human efforts. However, the exploration of machine learning (ML) techniques in varied domains has enabled the model parameterization in the drive system outside the laboratory settings. This paper presents an innovative use of a multi-agent reinforcement learning (MARL) approach for the parametrization of an LPTN model. First, a set of reinforcement learning agents are trained to estimate the optimized thermal parameters using the simulated data in several driving cycles (DCs). The selection of a reinforcement learning agent and the level of neurons in the RL model is made based on variability of the driving cycle data. Furthermore, transfer learning is performed on a new driving cycle data collected on the measurement setup. Statistical analysis and clustering techniques are proposed for the selection of an RL agent that has been pre-trained on the historical data. It is established that by synergizing within reinforcement learning techniques, it is possible to refine and adjust the RL learning models to effectively capture the complexities of thermal dynamics. The proposed MARL framework shows its capability to accurately reflect the motor’s thermal behavior under various driving conditions. The transfer learning usage in the proposed approach could yield significant improvement in the accuracy of temperature prediction in the new driving cycles data. This approach is proposed with the aim of developing more adaptive and efficient thermal management strategies for railway propulsion systems.

show abstract

Variational quantum circuit learning-enabled robust optimization for AI data center energy control and decarbonization

Ajagekar,

You

2024

Advances in Applied Energy

View full text Add to dashboard Cite

Deep reinforcement learning towards real-world dynamic thermal management of data centers

Cited by 12 publications

References 38 publications

Carbon Footprint Reduction for Sustainable Data Centers in Real-Time

Carbon Footprint Reduction for Sustainable Data Centers in Real-Time

Synergizing Transfer Learning and Multi-Agent Systems for Thermal Parametrization in Induction Traction Motors

Variational quantum circuit learning-enabled robust optimization for AI data center energy control and decarbonization

Contact Info

Product

Resources

About