Low-level autonomous control and tracking of quadrotor using reinforcement learning

Pi, Chen-Huan; Hu, Kai-Chun; Cheng, Stone; Wu, I‐Chen

doi:10.1016/j.conengprac.2019.104222

Cited by 68 publications

(32 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The method’s adaptability for multirotor UAVs is demonstrated. By contrast, the trained RL controllers presented in [ 17 , 18 , 21 ] can only be used for a specific multirotor with the same physical structure and parameters. In our method, the policy neural network output can be converted for each actuator unit according to the dynamic model of various geometric characteristics of the vehicle.…”

Section: Discussionmentioning

confidence: 99%

“…The relation between Equations ( 9 ) and ( 10 ) implies that given a policy

, a better policy

can be constructed by satisfying the inequality of Equation ( 9 ). Therefore, the proper estimation of

in Equation ( 9 ) is the major topic and the approach in this work is based on [ 18 ].…”

Section: Reinforcement Learning Algorithm and Implementationmentioning

confidence: 99%

“…In studies of Hwangbo et al [ 17 ] and Pi et al [ 18 ], they have introduced learning algorithms and implemented a control policy through RL training; this was shown to fully control a quadrotor by using a neural network to perform low-level stabilization and position control directly from the state quadrotor to four motor outputs. In Pi et al [ 19 ], a reinforcement learning with disturbance compensation was proposed using end-to-end manner to compensate wind gusts in outdoor environment.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

General Purpose Low-Level Reinforcement Learning Control for Multi-Axis Rotor Aerial Vehicles

Dai

et al. 2021

Sensors

Self Cite

View full text Add to dashboard Cite

This paper proposes a multipurpose reinforcement learning based low-level multirotor unmanned aerial vehicles control structure constructed using neural networks with model-free training. Other low-level reinforcement learning controllers developed in studies have only been applicable to a model-specific and physical-parameter-specific multirotor, and time-consuming training is required when switching to a different vehicle. We use a 6-degree-of-freedom dynamic model combining acceleration-based control from the policy neural network to overcome these problems. The UAV automatically learns the maneuver by an end-to-end neural network from fusion states to acceleration command. The state estimation is performed using the data from on-board sensors and motion capture. The motion capture system provides spatial position information and a multisensory fusion framework fuses the measurement from the onboard inertia measurement units for compensating the time delay and low update frequency of the capture system. Without requiring expert demonstration, the trained control policy implemented using an improved algorithm can be applied to various multirotors with the output directly mapped to actuators. The algorithm’s ability to control multirotors in the hovering and the tracking task is evaluated. Through simulation and actual experiments, we demonstrate the flight control with a quadrotor and hexrotor by using the trained policy. With the same policy, we verify that we can stabilize the quadrotor and hexrotor in the air under random initial states.

show abstract

Section: Discussionmentioning

confidence: 99%

“…The relation between Equations ( 9 ) and ( 10 ) implies that given a policy

, a better policy

can be constructed by satisfying the inequality of Equation ( 9 ). Therefore, the proper estimation of

in Equation ( 9 ) is the major topic and the approach in this work is based on [ 18 ].…”

Section: Reinforcement Learning Algorithm and Implementationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

General Purpose Low-Level Reinforcement Learning Control for Multi-Axis Rotor Aerial Vehicles

Dai

et al. 2021

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…This is one of the most promising areas of machine learning for autonomous control of vehicles. A model-free reinforcement algorithm developed by Chen-Huan Pi et al (2020) has a quad-copter tracing a predetermined path subjected disturbances. The algorithm rewards the system when the trajectory is as efficient at following the predetermined path subsequently minimising the error.…”

Section: Reinforcement Learningmentioning

confidence: 99%

Condition monitoring systems: a systematic literature review on machine-learning methods improving offshore-wind turbine operational management

Black

Richmond

Kolios

2021

International Journal of Sustainable Energy

View full text Add to dashboard Cite

Information is key. Offshore wind farms are installed with supervisory control and data acquisition systems (SCADA) gathering valuable information. Determining the precise condition of an asset is essential on achieving the expected operational lifetime and efficiency. Equipment fault detection is necessary to achieve this. This paper presents a systematic literature review of machine learning methods applied to condition monitoring systems, using both vibration information and SCADA data together. Starting with conventional methods using vibration models, such as Fast-Fourier transforms to five prominent supervised learning regression models; Artificial neural network, support vector regression, Bayesian network, random forest and K-nearest neighbour. This review specifically looks at how conventional vibration data can be combined with SCADA data to determine the assets condition.

show abstract

“…For an integrated electrical and heating system with wind energy, Zhang demonstrates PPO can optimize under power, demand, and spot electricity price uncertainties to minimize operational costs [29]. Actor-critic algorithms have also been used in optimizing costs for power systems, lot scheduling, hybrid vehicles, autonomous vehicles, brine injection, and drones [30][31][32][33][34][35][36].…”

Section: Literature Reviewmentioning

confidence: 99%

Comparing Reinforcement Learning Methods for Real-Time Optimization of a Chemical Process

2020

View full text Add to dashboard Cite

One popular method for optimizing systems, referred to as ANN-PSO, uses an artificial neural network (ANN) to approximate the system and an optimization method like particle swarm optimization (PSO) to select inputs. However, with reinforcement learning developments, it is important to compare ANN-PSO to newer algorithms, like Proximal Policy Optimization (PPO). To investigate ANN-PSO’s and PPO’s performance and applicability, we compare their methodologies, apply them on steady-state economic optimization of a chemical process, and compare their results to a conventional first principles modeling with nonlinear programming (FP-NLP). Our results show that ANN-PSO and PPO achieve profits nearly as high as FP-NLP, but PPO achieves slightly higher profits compared to ANN-PSO. We also find PPO has the fastest computational times, 10 and 10,000 times faster than FP-NLP and ANN-PSO, respectively. However, PPO requires more training data than ANN-PSO to converge to an optimal policy. This case study suggests PPO has better performance as it achieves higher profits and faster online computational times. ANN-PSO shows better applicability with its capability to train on historical operational data and higher training efficiency.

show abstract

Low-level autonomous control and tracking of quadrotor using reinforcement learning

Cited by 68 publications

References 17 publications

General Purpose Low-Level Reinforcement Learning Control for Multi-Axis Rotor Aerial Vehicles

General Purpose Low-Level Reinforcement Learning Control for Multi-Axis Rotor Aerial Vehicles

Condition monitoring systems: a systematic literature review on machine-learning methods improving offshore-wind turbine operational management

Comparing Reinforcement Learning Methods for Real-Time Optimization of a Chemical Process

Contact Info

Product

Resources

About