This work presents the application of reinforcement learning for the optimal resistive control of a point absorber. The model-free Q-learning algorithm is selected in order to maximise energy absorption in each sea state. Step changes are made to the controller damping, observing the associated penalty, for excessive motions, or reward, i.e. gain in associated power. Due to the general periodicity of gravity waves, the absorbed power is averaged over a time horizon lasting several wave periods. The performance of the algorithm is assessed through the numerical simulation of a point absorber subject to motions in heave in both regular and irregular waves. The algorithm is found to converge towards the optimal controller damping in each sea state. Additionally, the model-free approach ensures the algorithm can adapt to changes to the device hydrodynamics over time and is unbiased by modelling errors.

The levellised cost of energy of wave energy converters (WECs) is not competitive with fossil fuel-powered stations yet. To improve the feasibility of wave energy, it is necessary to develop effective control strategies that maximise energy absorption in mild sea states, whilst limiting motions in high waves. Due to their model-based nature, state-of-the-art control schemes struggle to deal with model uncertainties, adapt to changes in the system dynamics with time, and provide real-time centralised control for large arrays of WECs. Here, an alternative solution is introduced to address these challenges, applying deep reinforcement learning (DRL) to the control of WECs for the first time. A DRL agent is initialised from data collected in multiple sea states under linear model predictive control in a linear simulation environment. The agent outperforms model predictive control for high wave heights and periods, but suffers close to the resonant period of the WEC. The computational cost at deployment time of DRL is also much lower by diverting the computational effort from deployment time to training. This provides confidence in the application of DRL to large arrays of WECs, enabling economies of scale. Additionally, model-free reinforcement learning can autonomously adapt to changes in the system dynamics, enabling fault-tolerant control.

No abstract

No abstract

No abstract

No abstract

Unoccupied Underwater Vehicles (UUVs) are growing in importance and capabilities. Here, the trajectory control of an UUV carrying an object is investigated, with the consequent changes in system dynamics. For the first time, an Adaptive Model Predictive Control (AMPC) scheme for UUVs is developed, which selects optimal actions at the start of every time step to minimise the trajectory tracking error and prevent excessive changes in the control action over a receding time horizon. Prediction error minimisation is used to identify the linear model of the UUV in real time. The performance of AMPC is compared with existing PID and sliding-mode control (SMC) strategies through simulations. The latter is improved to prevent integral wind-up. While SMC results in best tracking performance, it imposes a strong burden on the motors due to its bang-bang action selection. AMPC presents smoother changes in applied thrust, but higher tracking errors due to non-linear effects and inaccuracies in the on-line system identification process. PID presents best overall performance, but its behaviour is expected to degrade on an actual ROV application due to sensor noise. This study will contribute to the selection of a suitable control scheme for future UUVs performing maintenance tasks autonomously.

To estimate the response of wave energy converters to different sea environments accurately is an ongoing challenge for researchers and industry, considering that there has to be a balance between guaranteeing their integrity whilst extracting the wave energy efficiently. For oscillating wave surge converters, the incident wave field is changed due to the pitching motion of the flap structure. A key component influencing this motion response is the Power Take-Off system used. Based on OpenFOAM, this paper includes the Power Take-off to establish a realistic model to simulate the operation of a three-dimensional oscillating wave surge converter by solving Reynolds Averaged Navier-Stokes equations. It examines the relationship between incident waves and the perturbed fluid field near the flap, which is of great importance when performing in arrays as neighbouring devices may influence each other. Furthermore, it investigates the influence of different control strategy systems (active and passive) in the energy extracted from regular waves related to the performance of the device. This system is estimated for each wave frequency considered and the results show the efficiency of the energy extracted from the waves is related to high amplitude pitching motions of the device in short periods of time.