Dynamic economic dispatch of power system based on DDPG algorithm

Liu, Zhicheng; Liu, Yipeng; Xu, Hui; Liao, Siyang; Zhu, Kefan; Jiang, Xinxiong

doi:10.1016/j.egyr.2022.02.231

Cited by 19 publications

(7 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As will be explained later, reinforcement learning is a suitable technique that could be applied to power plant operation, since it is based on rewarding desired actions and penalizes undesired ones so that the system learns autonomously the optimal action to be taken without running an optimizer each time of its operation [ 25 ]. In this context, Liu, Liu [ 26 ] presented a Deep Deterministic Policy Gradient (DDPG) model (which is one of the reinforcement learning algorithms that relies on neural networks) combining conventional generators, wind turbines, and solar PVs. The model aims to minimize the costs of power generation coming from the deviation from forecasted power penalties and the cost of fuel.…”

Section: Literature Reviewmentioning

confidence: 99%

“…The model aims to minimize the costs of power generation coming from the deviation from forecasted power penalties and the cost of fuel. When compared to a model predictive control approach, Liu, Liu [ 26 ] found that DDPP yields lower uncertainty cost, meaning a lower deviation from the forecast. However, the method is not compared with optimization and did not include heat supply which may complicate the modeling.…”

Section: Literature Reviewmentioning

confidence: 99%

See 1 more Smart Citation

Assessing and comparing a DDPG model and GA optimization for a heat and power virtual power plant operating in a power purchase agreement scheme

Elgamal,

Shahrestani,

Vahdati

2024

Heliyon

View full text Add to dashboard Cite

Section: Literature Reviewmentioning

confidence: 99%

Section: Literature Reviewmentioning

confidence: 99%

Assessing and comparing a DDPG model and GA optimization for a heat and power virtual power plant operating in a power purchase agreement scheme

Elgamal,

Shahrestani,

Vahdati

2024

Heliyon

View full text Add to dashboard Cite

“…16 The DDPG algorithm employs an Actor-Critic network to approximate the policy function 𝜇 and utilizes the DQN algorithm to train the network function Q, which enables the computation of temporal difference errors and the implementation of gradient updates from the Online Network to the Target Network. 17 Q function in the Critic network represents the expected value R t obtained after executing action a t output by actor network and policy 𝜇 in state s t , with a discount factor of 𝛾:…”

Section: Ddpg-based Control Algorithmmentioning

confidence: 99%

“…Deep deterministic policy gradient was proposed by the DeepMind team in 2016 as a strategy algorithm that incorporates deep learning neural networks into DPG 16 . The DDPG algorithm employs an Actor‐Critic network to approximate the policy function

\mu

and utilizes the DQN algorithm to train the network function Q, which enables the computation of temporal difference errors and the implementation of gradient updates from the Online Network to the Target Network 17 …”

Section: Ddpg‐based Control Algorithmmentioning

confidence: 99%

Optimization control of the double‐capacity water tank‐level system using the deep deterministic policy gradient algorithm

Pei

2023

Engineering Reports

View full text Add to dashboard Cite

Process control systems are subject to external factors such as changes in working conditions and perturbation interference, which can significantly affect the system's stability and overall performance. The application and promotion of intelligent control algorithms with self‐learning, self‐optimization, and self‐adaption characteristics have thus become a challenging yet meaningful research topic. In this article, we propose a novel approach that incorporates the deep deterministic policy gradient (DDPG) algorithm into the control of double‐capacity water tanklevel system. Specifically, we introduce a fully connected layer on the observer side of the critic network to enhance its expression capability and processing efficiency, allowing for the extraction of important features for water‐level control. Additionally, we optimize the node parameters of the neural network and use the RELU activation function to ensure the network's ability to continuously observe and learn from the external water tank environment while avoiding the issue of vanishing gradients. We enhance the system's feedback regulation ability by adding the PID controller output to the observer input based on the liquid level deviation and height. This integration with the DDPG control method effectively leverages the benefits of both, resulting in improved robustness and adaptability of the system. Experimental results show that our proposed model outperforms traditional control methods in terms of convergence, tracking, anti‐disturbance and robustness performances, highlighting its effectiveness in improving the stability and precision of double‐capacity water tank systems.

show abstract

“…Zhang et al proposed a deep-reinforcement-learning-based energy scheduling strategy to optimize multiple targets, taking diversified uncertainties into account; an integrated power, heat, and natural gas system consisting of energy-coupling units and wind power generation interconnected via a power grid was modeled as a Markov decision process [ 35 ]. Liu et al proposed an adaptive uncertain dynamic economic dispatch method based on deep deterministic policy gradient (DDPG); on the basis of the economic dispatch model, they built a Markov decision process for power systems [ 36 ]. In this paper, the operation optimization of the sugarcane milling process is described as an MDP process, which is modeled as follows:…”

Section: Solving the Collaborative Optimization Model Of Mf-ef-if In ...mentioning

confidence: 99%

Multi-Objective Optimization of Sugarcane Milling System Operations Based on a Deep Data-Driven Model

Chen

Meng

et al. 2022

Foods

View full text Add to dashboard Cite

The extraction of sugarcane juice is the first step of sugar production. The optimal values of process indicators and the set values of operating parameters in this process are still determined by workers’ experience, preventing adaptive adjustment of the production process. To address this issue, a multi-objective optimization framework based on a deep data-driven model is proposed to optimize the operation of sugarcane milling systems. First, the sugarcane milling process is abstracted as the interaction of material flow, energy flow, and information flow (MF–EF–IF) by introducing synergetic theory, and each flow’s order parameters and state parameters are obtained. Subsequently, the state parameters of the subsystems are taken as inputs, and the order parameters—including the grinding capacity, electric consumption per ton of sugarcane, and sucrose extraction—are produced as outputs. A collaborative optimization model of the MF–EF–IF of the milling system is established by using a deep kernel extreme learning machine (DK-ELM). The established milling system model is applied for an improved multi-objective chicken swarm optimization (IMOCSO) algorithm to obtain the optimal values of the order parameters. Finally, the milling process is described as a Markov decision process (MDP) with the optimal values of the order parameters as the control objectives, and an improved deep deterministic policy gradient (DDPG) algorithm is employed to achieve the adaptive optimization of the operating parameters under different working conditions of the milling system. Computational experiments indicate that enhanced performance is achieved, with an increase of 3.2 t per hour in grinding capacity, a reduction of 660 W per ton in sugarcane electric consumption, and an increase of 0.03% in the sucrose extraction.

show abstract

Dynamic economic dispatch of power system based on DDPG algorithm

Cited by 19 publications

References 6 publications

Assessing and comparing a DDPG model and GA optimization for a heat and power virtual power plant operating in a power purchase agreement scheme

Assessing and comparing a DDPG model and GA optimization for a heat and power virtual power plant operating in a power purchase agreement scheme

Optimization control of the double‐capacity water tank‐level system using the deep deterministic policy gradient algorithm

Multi-Objective Optimization of Sugarcane Milling System Operations Based on a Deep Data-Driven Model

Contact Info

Product

Resources

About