A Deep Reinforcement Learning Motion Control Strategy of a Multi-rotor UAV for Payload Transportation with Minimum Swing

Panetsos, Fotis; Karras, George C.; Kyriakopoulos, Kostas J.

doi:10.1109/med54222.2022.9837220

Cited by 10 publications

(5 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, in [12] the chosen discount factor is close to 1. Li et al [57] and Hu and Wang [53] chose a value of 0.9, and Castro et al [66] and Panetsos et al [60] chose a discount factor value of 0.99.…”

Section: Value-function-based Algorithmsmentioning

confidence: 99%

See 1 more Smart Citation

A Survey of Offline- and Online-Learning-Based Algorithms for Multirotor Uavs

Sönmez,

Rutherford,

Valavanis

2024

Drones

View full text Add to dashboard Cite

Multirotor UAVs are used for a wide spectrum of civilian and public domain applications. Their navigation controllers include onboard sensor suites that facilitate safe, autonomous or semi-autonomous multirotor flight, operation, and functionality under nominal and detrimental conditions and external disturbances, even when flying in uncertain and dynamically changing environments. During the last decade, given the available computational power, different learning-based algorithms have been derived, implemented, and tested to navigate and control, among other systems, multirotor UAVs. Learning algorithms have been and are used to derive data-driven based models, to identify parameters, to track objects, to develop navigation controllers, and to learn the environments in which multirotors operate. Learning algorithms combined with model-based control techniques have proven beneficial when applied to multirotors. This survey summarizes the research published since 2015, dividing algorithms, techniques, and methodologies into offline and online learning categories and then further classifying them into machine learning, deep learning, and reinforcement learning sub-categories. An integral part and focus of this survey is on online learning algorithms as applied to multirotors, with the aim to register the type of learning techniques that are either hard or almost hard real-time implementable, as well as to understand what information is learned, why, how, and how fast. The outcome of the survey offers a clear understanding of the recent state of the art and of the type and kind of learning-based algorithms that may be implemented, tested, and executed in real time.

show abstract

Section: Value-function-based Algorithmsmentioning

confidence: 99%

“…Panetsos et al [60] offer a solution to the payload transportation challenge using a DRL approach. An attitude PID controller is used in the inner loop of the cascaded controller structure, while a position controller in the outer loop is replaced with a TD3-based DRL algorithm.…”

Section: Actor-critic Algorithmsmentioning

confidence: 99%

A Survey of Offline- and Online-Learning-Based Algorithms for Multirotor Uavs

Sönmez,

Rutherford,

Valavanis

2024

Drones

View full text Add to dashboard Cite

show abstract

“…No connection between UAV and ground 47,48,50,51,[53][54][55][56][58][59][60][61][62][63][64][65][66][67][68][69]71,[74][75][76][77][78][79][80][81][82][83][86][87][88]90,91,129,134,137,140,143] Connection for mechanical purposes during part of the operation [193] (only during recovery of payload) [119,131] Very high operating altitude (>1 km) [147,148,175] Short-term missions (inspection) [133,163]…”

Section: Reason For Not Transferring Power Over the Tether Publicationsmentioning

confidence: 99%

Tethered Unmanned Aerial Vehicles—A Systematic Review

et al. 2023

View full text Add to dashboard Cite

In recent years, there has been a remarkable surge in the development and research of tethered aerial systems, thus reflecting a growing interest in their diverse applications. Long-term missions involving aerial vehicles present significant challenges due to the limitations of current battery solutions. Tethered vehicles can circumvent such restrictions by receiving their power from an element on the ground such as a ground station or a mobile terrestrial platform. Tethered Unmanned Aerial Vehicles (UAVs) can also be applied to load transportation achieved by a single or multiple UAVs. This paper presents a comprehensive systematic literature review, with a special focus on solutions published in the last five years (2017–2022). It emphasizes the key characteristics that are capable of grouping publications by application scope, propulsion method, energy transfer solution, perception sensors, and control techniques adopted. The search was performed in six different databases, thereby resulting in 1172 unique publications, from which 182 were considered for inclusion in the data extraction phase of this review. Among the various aircraft types, multirotors emerged as the most widely used category. We also identified significant variations in the application scope of tethered UAVs, thus leading to tailored approaches for each use case, such as the fixed-wing model being predominant in the wind generation application and the lighter-than-air aircraft in the meteorology field. Notably, the classical Proportional–Integral–Derivative (PID) control scheme emerged as the predominant control methodology across the surveyed publications. Regarding energy transfer techniques, most publications did not explicitly describe their approach. However, among those that did, high-voltage DC energy transfer emerged as the preferred solution. In summary, this systematic literature review provides valuable insights into the current state of tethered aerial systems, thereby showcasing their potential as a robust and sustainable alternative to address the challenges associated with long-duration aerial missions and load transportation.

show abstract

“…Model-free reinforcement learning (RL) has shown to have benefits in these situations as its sample-based properties allow for robust trajectory generation even if the model of the system is unknown [7]. Only a few RL-based solutions for the specific case of a RUAV and suspended payload are found in the literature [9,10,11,12,13].…”

Section: Introductionmentioning

confidence: 99%

“…The state-of-the-art deep RL algorithm for continuous autonomous control is currently the twin-delayed deep deterministic policy gradient (TD3) [14]. This algorithm has been applied to RUAV navigation [15,16], but, to the best of our knowledge, only one application to a multi-rotor with a suspended payload is available [13]. The TD3 agent is used to replace the position PID controller as a swing-minimizing controller and shows good tracking of a variety of 3D waypoints with significant swing reduction.…”

Section: Introductionmentioning

confidence: 99%

Model-Free trajectory planning for a rotary-wing unmanned aerial vehicle with an uncertain suspended payload

Hohbach,

Jordaan,

Engelbrecht

2023

MATEC Web Conf.

View full text Add to dashboard Cite

Payload transport using rotary-wing unmanned aerial vehicles (RUAVs) has grown in popularity. Suspending the payload from the RUAV expands the range of use cases but does so at the expense of changing the dynamics that may result in instability. This paper explores how a model-free solution can be used to plan robust trajectories that account for the added oscillations if the payload is varied. The twin-delayed deep deterministic policy gradient (TD3) algorithm is a model-free reinforcement learning method and is used to train an agent to be implemented as a local planner to plan optimal trajectories that can complete two defined circuits whilst minimizing the swing of the payload. A non-linear model predictive controller (NMPC) is implemented as a model-based solution to evaluate the capabilities of the model-free results determining its viability, without choosing a superior approach. The results indicate that the model-free TD3 agent has comparable results to the model-based NMPC for two defined circuits, with increased robustness to payload uncertainty when trained with different payload parameters.

show abstract

A Deep Reinforcement Learning Motion Control Strategy of a Multi-rotor UAV for Payload Transportation with Minimum Swing

Cited by 10 publications

References 18 publications

A Survey of Offline- and Online-Learning-Based Algorithms for Multirotor Uavs

A Survey of Offline- and Online-Learning-Based Algorithms for Multirotor Uavs

Tethered Unmanned Aerial Vehicles—A Systematic Review

Model-Free trajectory planning for a rotary-wing unmanned aerial vehicle with an uncertain suspended payload

Contact Info

Product

Resources

About