Data Centers Job Scheduling with Deep Reinforcement Learning

Liang, Sisheng; Yang, Zhou; Chen, Yong

doi:10.1007/978-3-030-47436-2_68

Cited by 20 publications

(13 citation statements)

References 13 publications

(21 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The benefits brought by DRL to other fields such as robotics and computer games are also employed to solve complex decision-making problems. Some recent works use DRL to solve some of the most prominent complex optimization problems such as resource management problems (Mao et al 2016), job scheduling (Chen et al 2017;Liu et al 2020;Liang et al 2020), VRP (Nazari et al 2018;Lin et al 2020;Zhao et al 2020;Yu et al 2019), and production scheduling problems (Waschneck et al 2018b, a;Hubbs et al 2020).…”

Section: Literature Reviewmentioning

confidence: 99%

A deep reinforcement learning assisted simulated annealing algorithm for a maintenance planning problem

2022

View full text Add to dashboard Cite

Maintenance planning aims to improve the reliability of assets, prevent the occurrence of asset failures, and reduce maintenance costs associated with downtime of assets and maintenance resources (such as spare parts and workforce). Thus, effective maintenance planning is instrumental in ensuring high asset availability with the minimum cost. Nevertheless, to find such optimal planning is a nontrivial task due to the (i) complex and usually nonlinear inter-relationship between different planning decisions (e.g., inventory level and workforce capacity), and (ii) stochastic nature of the system (e.g., random failures of parts installed in assets). To alleviate these challenges, we study a joint maintenance planning problem by considering several decisions simultaneously, including workforce planning, workforce training, and spare parts inventory management. We develop a hybrid solution algorithm ($$\mathcal {DRLSA}$$ DRLSA ) that is a combination of Double Deep Q-Network based Deep Reinforcement Learning (DRL) and Simulated Annealing (SA) algorithms. In each episode of the proposed algorithm, the best solution found by DRL is delivered to SA to be used as an initial solution, and the best solution of SA is delivered to DRL to be used as the initial state. Different from the traditional SA algorithms where neighborhood structures are selected only randomly, the DRL part of $$\mathcal {DRLSA}$$ DRLSA learns to choose the best neighborhood structure to use based on experience gained from previous episodes. We compare the performance of the proposed solution algorithm with several well-known meta-heuristic algorithms, including Simulated Annealing, Genetic Algorithm (GA), and Variable Neighborhood Search (VNS). Further, we also develop a Machine Learning (ML) algorithm (i.e., K-Median) as another benchmark in which different properties of spare parts (e.g., failure rates, holding costs, and repair rates) are used as clustering features for the ML algorithm. Our study reveals that the $$\mathcal {DRLSA}$$ DRLSA finds the optimal solutions for relatively small-size instances, and it has the potential to outperform traditional meta-heuristic and ML algorithms.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

A deep reinforcement learning assisted simulated annealing algorithm for a maintenance planning problem

2022

View full text Add to dashboard Cite

show abstract

“…Job dispatching in scheduling problems. Job scheduling is extensive studied from both theoretical [9], [10], [11], [12], [13], [14], [15], [29] and system-level perspectives [30], [31], [32], [33], [34], [35]. The theoretical works usually formulate the job completion time (JCT) minimization problems as combinatorial, constrained optimizaiton problems and solves them with various approaches, especially the approximate algorithms.…”

Section: Related Workmentioning

confidence: 99%

“…Simplistically, job scheduling studies when to dispatch each job to which server. A majority of job scheduling algorithms have been proposed by formulating combinatorial optimization problems with scenario-oriented constraints [9], [10], [11], [12], [13], [14], [15]. To solve the combinatorial programs, algorithms are designed based on various theoretical approaches, includ-• H. Zhao, S. Deng, F. Chen, and J. Yin are with the College of Computer Science and Technology, Zhejiang University, Hangzhou 310058, China.…”

Section: Introductionmentioning

confidence: 99%

“…e-mail: dustdar@dsg.tuwien.ac.at • A. Y. Zomaya is with the School of Computer Science, University of Sydney, Sydney, NSW 2006, Australia. e-mail: albert.zomaya@sydney.edu.au ing relaxed integer programming [10], online primal-dual approaches [11], heuristics [12], [13], deep reinforcement learning (DRL) [14], [15], etc. Correspondingly, load balancing studies how to dispatch the traffic of service requests to the backend servers with the purpose of minimizing the mean response time.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning to Schedule Multi-Server Jobs with Fluctuated Processing Speeds

Hong-xia¹,

Deng²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

Multi-server jobs are imperative in modern cloud computing systems. A multi-server job has multiple components and requests multiple servers for being served. How to allocate restricted computing devices to jobs is a topic of great concern, which leads to the job scheduling and load balancing algorithms thriving. However, current job dispatching algorithms require the service rates to be changeless and knowable, which is difficult to realize in production systems. Besides, for multi-server jobs, the dispatching decision for each job component follows the All-or-Nothing property under service locality constraints and resource capacity limits, which is not well supported by mainstream algorithms. In this paper, we propose a dispatching algorithm for multi-server jobs that learns the unknown service rates and simultaneously maximizes the expected Accumulative Social Welfare (Asw). We formulate the Asw as the sum of utilities of jobs and servers achieved over each time slot. The utility of a job is proportional to the valuation for being served, which is mainly impacted by the fluctuating but unknown service rates. We maximize the Asw without knowing the exact valuations, but approximate them with exploration-exploitation. From this, we bring in several evolving statistics and maximize the statistical Asw with dynamic programming. The proposed algorithm is proved to have a polynomial complexity and a State-of-the-Art regret. We validate it with extensive simulations and the results show that the proposed algorithm outperforms several benchmark policies with improvements by up to 73%, 36%, and 28%, respectively.

show abstract

“…While its capabilities have often been demonstrated by learning policies for video games, such as Atari games [52], Starcraft [80], and Dota [8], it can be applied to a wide variety of real-world scenarios. For example, RL has been used for scheduling in data centers [42] and for adaptive power management [44,27,91,12]. Furthermore, reinforcement learning can also be applied to many problems in the domain of robotics, for example, navigation [9] or robotic manipulation [55].…”

Section: Introductionmentioning

confidence: 99%

A Survey of Domain-Specific Architectures for Reinforcement Learning

Rothmann

Porrmann

2022

IEEE Access

View full text Add to dashboard Cite

Reinforcement learning algorithms have been very successful at solving sequential decisionmaking problems in many different problem domains. However, their training is often time-consuming, with training times ranging from multiple hours to weeks. The development of domain-specific architectures for reinforcement learning promises faster computation times, decreased experiment turn-around time, and improved energy efficiency. This paper presents a review of hardware architectures for the acceleration of reinforcement learning algorithms. FPGA-based implementations are the focus of this work, but GPU-based approaches are considered as well. Both tabular and deep reinforcement learning algorithms are included in this survey. The techniques employed in different implementations are highlighted and compared. Finally, possible areas for future work are suggested, based on the preceding discussion of existing architectures.

show abstract

Data Centers Job Scheduling with Deep Reinforcement Learning

Cited by 20 publications

References 13 publications

A deep reinforcement learning assisted simulated annealing algorithm for a maintenance planning problem

A deep reinforcement learning assisted simulated annealing algorithm for a maintenance planning problem

Learning to Schedule Multi-Server Jobs with Fluctuated Processing Speeds

A Survey of Domain-Specific Architectures for Reinforcement Learning

Contact Info

Product

Resources

About