Real-world sequential decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives. Despite this, the majority of research in reinforcement learning and decision-theoretic planning either assumes only a single objective, or that multiple objectives can be adequately handled via a simple linear combination. Such approaches may oversimplify the underlying problem and hence produce suboptimal results. This paper serves as a guide to the application of multi-objective methods to difficult problems, and is aimed at researchers who are already familiar with single-objective reinforcement learning and planning methods who wish to adopt a multi-objective perspective on their research, as well as practitioners who encounter multi-objective decision problems in practice. It identifies the factors that may influence the nature of the desired solution, and illustrates by example how these influence the design of multi-objective decision-making systems for complex problems.

Abstract. This paper presents the state-of-the-art technologies and development trends of wind turbine drivetrains – the system that converts kinetic energy of the wind to electrical energy – in different stages of their life cycle: design, manufacturing, installation, operation, lifetime extension, decommissioning and recycling. Offshore development and digitalization are also a focal point in this study. Drivetrain in this context includes the whole power conversion system: main bearing, shafts, gearbox, generator and power converter. The main aim of this article is to review the drivetrain technology development as well as to identify future challenges and research gaps. The main challenges in drivetrain research identified in this paper include drivetrain dynamic responses in large or floating turbines, aerodynamic and farm control effects, use of rare-earth material in generators, improving reliability through prognostics, and use of advances in digitalization. These challenges illustrate the multidisciplinary aspect of wind turbine drivetrains, which emphasizes the need for more interdisciplinary research and collaboration.

Outbreaks of SARS-CoV-2 are threatening the health care systems of several countries around the world. The initial control of SARS-CoV-2 epidemics relied on non-pharmaceutical interventions, such as social distancing, teleworking, mouth masks and contact tracing. However, as pre-symptomatic transmission remains an important driver of the epidemic, contact tracing efforts struggle to fully control SARS-CoV-2 epidemics. Therefore, in this work, we investigate to what extent the use of universal testing, i.e., an approach in which we screen the entire population, can be utilized to mitigate this epidemic. To this end, we rely on PCR test pooling of individuals that belong to the same households, to allow for a universal testing procedure that is feasible with the limited testing capacity. We evaluate two isolation strategies: on the one hand pool isolation, where we isolate all individuals that belong to a positive PCR test pool, and on the other hand individual isolation, where we determine which of the individuals that belong to the positive PCR pool are positive, through an additional testing step. We evaluate this universal testing approach in the STRIDE individual-based epidemiological model in the context of the Belgian COVID-19 epidemic. As the organisation of universal testing will be challenging, we discuss the different aspects related to sample extraction and PCR testing, to demonstrate the feasibility of universal testing when a decentralized testing approach is used. We show through simulation, that weekly universal testing is able to control the epidemic, even when many of the contact reductions are relieved. Finally, our model shows that the use of universal testing in combination with stringent contact reductions could be considered as a strategy to eradicate the virus.

Wind farms are an indispensable driver toward renewable and nonpolluting energy resources. However, as ideal sites are limited, placement in remote and challenging locations results in higher logistics costs and lower average wind speeds. Therefore, it is critical to increase the reliability of the turbines to reduce maintenance costs. Robust implementation requires a thorough understanding of the loads subject to the turbine's control. Yet, such dynamically changing multidimensional loads are uncommon with other machinery, and generally underresearched. Therefore, a multitiered approach is proposed to investigate the load spectrum occurring in wind farms. Our approach relies on both fundamental research using controllable test rigs, as well as analyses of real-world loading conditions in high-frequency supervisory control and data acquisition data. A method is introduced to detect operational zones in wind farm data and link them with load distributions. Additionally, while focused research further investigates the load spectrum, a method is proposed that continuously optimizes the farm's control protocols without the need to fully understand the loads that occur. A case of gearbox failure is investigated based on a vast body of past experiments and suspect

Multi-agent coordination is prevalent in many real-world applications. However, such coordination is challenging due to its combinatorial nature. An important observation in this regard is that agents in the real world often only directly affect a limited set of neighbouring agents. Leveraging such loose couplings among agents is key to making coordination in multi-agent systems feasible. In this work, we focus on learning to coordinate. Specifically, we consider the multi-agent multi-armed bandit framework, in which fully cooperative loosely-coupled agents must learn to coordinate their decisions to optimize a common objective. We propose multi-agent Thompson sampling (MATS), a new Bayesian exploration-exploitation algorithm that leverages loose couplings. We provide a regret bound that is sublinear in time and low-order polynomial in the highest number of actions of a single agent for sparse coordination graphs. Additionally, we empirically show that MATS outperforms the state-of-the-art algorithm, MAUCE, on two synthetic benchmarks, and a novel benchmark with Poisson distributions. An example of a loosely-coupled multi-agent system is a wind farm. Coordination within the wind farm is necessary to maximize power production. As upstream wind turbines only affect nearby downstream turbines, we can use MATS to efficiently learn the optimal control mechanism for the farm. To demonstrate the benefits of our method toward applications we apply MATS to a realistic wind farm control task. In this task, wind turbines must coordinate their alignments with respect to the incoming wind vector in order to optimize power production. Our results show that MATS improves significantly upon state-of-the-art coordination methods in terms of performance, demonstrating the value of using MATS in practical applications with sparse neighbourhood structures.

No abstract

Epidemics of infectious diseases are an important threat to public health and global economies. Yet, the development of prevention strategies remains a challenging process, as epidemics are non-linear and complex processes. For this reason, we investigate a deep reinforcement learning approach to automatically learn prevention strategies in the context of pandemic influenza. Firstly, we construct a new epidemiological meta-population model, with 379 patches (one for each administrative district in Great Britain), that adequately captures the infection process of pandemic influenza. Our model balances complexity and computational efficiency such that the use of reinforcement learning techniques becomes attainable. Secondly, we set up a ground truth such that we can evaluate the performance of the "Proximal Policy Optimization" algorithm to learn in a single district of this epidemiological model. Finally, we consider a large-scale problem, by conducting an experiment where we aim to learn a joint policy to control the districts in a community of 11 tightly coupled districts, for which no ground truth can be established. This experiment shows that deep reinforcement learning can be used to learn mitigation policies in complex epidemiological models with a large state and action space.

Abstract. This paper presents the state-of-the-art technologies and development trends of wind turbine drivetrains – the energy conversion systems transferring the kinetic energy of the wind to electrical energy – in different stages of their life cycle: design, manufacturing, installation, operation, lifetime extension, decommissioning, and recycling. Offshore development and digitalization are also a focal point in this study. The main aim of this article is to review the drivetrain technology development as well as to identify future challenges and research gaps. Drivetrain in this context includes the whole power conversion system: main bearing, shafts, gearbox, generator, and power converter. The paper discusses current design technologies for each component along with advantages and disadvantages. The discussion of the operation phase highlights the condition monitoring methods currently employed by the industry as well as emerging areas. This article also illustrates the multidisciplinary aspect of wind turbine drivetrains, which emphasizes the need for more interdisciplinary research and collaboration.