Data-Driven Wind Farm Control via Multiplayer Deep Reinforcement Learning

Dong, Hongyang; Zhao, Xiaowei

doi:10.1109/tcst.2022.3223185

Cited by 16 publications

(5 citation statements)

References 24 publications

(80 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent papers, the influence of each turbine on its neighbours via these edges has been investigated through physicsinduced edge weighting [23] and attention networks [24]. Dong & Zhao [25] used a graph-style approach to split a large wind farm into sub-groups to apply more localised WFC methods, but they did not train a GNN itself. The specific form of GNN chosen in the current work was the Graph Convolutional Network (GCN) [26], where each layer of the network has trainable weights that are applied to the combination of node features obtained through message passing via the (weighted) edges.…”

Section: Graph-ddpgmentioning

confidence: 99%

Graph-based Deep Reinforcement Learning for Wind Farm Set-Point Optimisation

Sheehan,

Poole,

Silva Filho

et al. 2024

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

Wake steering is a form of wind farm control in which upstream turbines are deliberately yawed to misalign with the free-stream wind in order to prevent their wakes from impacting turbines further downstream. This technique can give a net increase in power generated by an array of turbines compared to greedy control, but the optimisation of multiple turbine set-points under varying wind conditions can be infeasibly complex for traditional, white-box models. In this work, a novel deep reinforcement learning method combining the standard Deep Deterministic Policy Gradient algorithm with a graph representation of potential inter-turbine wake connections was trained to apply wake steering to an array of nine turbines under varying wind directions. The method demonstrated strong performance for wind directions with large potential farm power gains. A steady-state wind farm solver was used, employing a “quasi-dynamic” approach to sampling wind directions, to achieve an additional 47 MW (6.5%) power over four wind directions compared to greedy control.

show abstract

Section: Graph-ddpgmentioning

confidence: 99%

Graph-based Deep Reinforcement Learning for Wind Farm Set-Point Optimisation

Sheehan,

Poole,

Silva Filho

et al. 2024

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

show abstract

“…Several RL algorithms can be applied to wind farm control problems, but recent work has focused on actor-critic approaches [12]: Deep Deterministic Policy Gradient (DDPG) is used in [13][14][15][16], Proximal Policy Gradient (PPO) in [8], and a custom actor-critic method is developed by [17]. In the following, we briefly introduce this family of algorithms.…”

Section: Introductionmentioning

confidence: 99%

Towards fine tuning wake steering policies in the field: an imitation-based approach

Bizon Monroc,

Bušić,

Dubuc

et al. 2024

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

Yaw misalignment strategies can increase the power output of wind farms by mitigating wake effects, but finding optimal yaws requires overcoming both modeling errors and the growing complexity of the problem as the size of the farm grows. Recent works have therefore proposed decentralized multi-agent reinforcement learning (MARL) as a model-free, data-based alternative to learn online. These solutions have led to significant increases in total power production on experiments with both static and dynamic wind farms simulators. Yet experiments in dynamic simulations suggest that convergence time remains too long for online learning on real wind farms. As an improvement, baseline policies obtained by optimizing offline through steady-state models can be fed as inputs to an online reinforcement learning algorithm. This method however does not guarantee a smooth transfer of the policies to the real wind farm. This is aggravated when using function approximation approaches such as multi-layer neural networks to estimate policies and value functions. We propose an imitation approach, where learning a policy is first considered a supervised learning problem by deriving references from steady-state wind farm models, and then as an online reinforcement learning task for adaptation in the field. This approach leads to significant increases in the amount of energy produced over a lookup table (LUT) baseline on experiments done with the mid-fidelity dynamic simulator FAST.Farm under both static and varying wind conditions.

show abstract

“…As this subdivision to different tasks results in a similarly nested architecture, some recent examples of applications to cascaded structures exist in [25], where hierarchical RL is used to control the water level in connected canals and [26] to optimize fuel cell use and degradation in a fuel cell hybrid electrical vehicle. Next to hierarchical RL, multi-agent RL [27], in which the simultaneous control of RL agents is studied, finds applications in systems with a degree of cascaded structure, such as smart grids and industrial production energy balancing [28] or wake control of wind farms [29], by considering all controlled variables as distinct subsystems. Though the challenge of a cascaded control structure has surfaced in a number of works on reinforcement learning, to the best of our knowledge, the potential of RL for such systems has not been studied distinctly.…”

Section: Introductionmentioning

confidence: 99%

Optimizing Cascaded Control of Mechatronic Systems through Constrained Residual Reinforcement Learning

2023

View full text Add to dashboard Cite

Cascaded control structures are prevalent in industrial systems with many disturbances to obtain stable control but are cumbersome and challenging to tune. In this work, we propose cascaded constrained residual reinforcement learning (RL), an intuitive method that allows to improve the performance of a cascaded control structure while maintaining safe operation at all times. We draw inspiration from the constrained residual RL framework, in which a constrained reinforcement learning agent learns corrective adaptations to a base controller’s output to increase optimality. We first revisit the interplay between the residual agent and the baseline controller and subsequently extend this to the cascaded case. We analyze the differences and challenges this structure brings and derive some principle insights from this into the stability and operation of the cascaded residual architecture. Next, we propose a novel actor structure to enable efficient learning under the cascaded setting. We show that the standard algorithm is suboptimal for application to cascaded control structures and validate our method on a high-fidelity simulator of a dual motor drivetrain, resulting in a performance improvement of 14.7% on average, with only a minor decrease in performance occurring during the training phase. We study the different principles constituting the method and examine and validate their contribution to the algorithm’s performance under the considered cascaded control structure.

show abstract

Data-Driven Wind Farm Control via Multiplayer Deep Reinforcement Learning

Abstract: How to cite:Please refer to published version for the most recent bibliographic citation information. If a published version is known of, the repository item page linked to above, will contain details on accessing it.

Cited by 16 publications

References 24 publications

Graph-based Deep Reinforcement Learning for Wind Farm Set-Point Optimisation

Graph-based Deep Reinforcement Learning for Wind Farm Set-Point Optimisation

Towards fine tuning wake steering policies in the field: an imitation-based approach

Optimizing Cascaded Control of Mechatronic Systems through Constrained Residual Reinforcement Learning

Contact Info

Product

Resources

About