AUV Position Tracking Control Using End-to-End Deep Reinforcement Learning

Carlucho, Ignacio; Paula, Mariano De; Wang, Sen; Menna, Bruno V.; Pétillot, Yvan; Acosta, Gerardo

doi:10.1109/oceans.2018.8604791

Cited by 37 publications

(21 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Other work has used the same algorithm for quadrotor UAV attitude control [ 23 ]. In [ 24 ], navigation of an autonomous underwater vehicle (AUV) was addressed using the deep deterministic policy gradients (DDPG) algorithm. Other work has used the same technique (DDPG) for landing a quadrotor in a moving object [ 25 ].…”

Section: Related Workmentioning

confidence: 99%

Deep Reinforcement Learning for End-to-End Local Motion Planning of Autonomous Aerial Robots in Unknown Outdoor Environments: Real-Time Flight Experiments

Doukhi

Lee

2021

Sensors

View full text Add to dashboard Cite

Autonomous navigation and collision avoidance missions represent a significant challenge for robotics systems as they generally operate in dynamic environments that require a high level of autonomy and flexible decision-making capabilities. This challenge becomes more applicable in micro aerial vehicles (MAVs) due to their limited size and computational power. This paper presents a novel approach for enabling a micro aerial vehicle system equipped with a laser range finder to autonomously navigate among obstacles and achieve a user-specified goal location in a GPS-denied environment, without the need for mapping or path planning. The proposed system uses an actor–critic-based reinforcement learning technique to train the aerial robot in a Gazebo simulator to perform a point-goal navigation task by directly mapping the noisy MAV’s state and laser scan measurements to continuous motion control. The obtained policy can perform collision-free flight in the real world while being trained entirely on a 3D simulator. Intensive simulations and real-time experiments were conducted and compared with a nonlinear model predictive control technique to show the generalization capabilities to new unseen environments, and robustness against localization noise. The obtained results demonstrate our system’s effectiveness in flying safely and reaching the desired points by planning smooth forward linear velocity and heading rates.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep Reinforcement Learning for End-to-End Local Motion Planning of Autonomous Aerial Robots in Unknown Outdoor Environments: Real-Time Flight Experiments

Doukhi

Lee

2021

Sensors

View full text Add to dashboard Cite

show abstract

“…O problema da navegac ¸ão sem mapa para UAVs e UUVs já foi abordado em vários trabalhos. Rodriguez et al [Rodriguez-Ramos et al 2018] e Carlucho et al [Carlucho et al 2018] são os principais exemplos para cada um dos respectivos veículos.…”

Section: Trabalhos Relacionadosunclassified

Aprendizado por Reforço Profundo para Navegação sem Mapa de um Veículo Híbrido Aéreo-Aquático

Grando¹,

Drews-Jr²

2021

Anais Estendidos Do XIII Simpósio Brasileiro De Robótica E XVIII Simpósio Latino Americano De Robótica (SBR/LARS Estendido 2021

View full text Add to dashboard Cite

Realizar a navegação entre meios é uma tarefa desafiadora para robôs móveis híbridos, especialmente em cenários com obstáculos. Este trabalho apresenta uma abordagem baseada em aprendizado por reforço profundo (Deep-RL) para navegação autônoma de um tipo específico de robô móvel híbrido: um Veículo Híbrido Tipo Ar-Água (HUAUV). A abordagem proposta utiliza somente informação de sensores de distância e de informações relativas à localização do veículo para realizar a navegação. Resultados obtidos mostram que é possível realizar navegação sem mapa do início ao fim sem usar nenhum tipo de operação manual, somente utilizando os agentes baseados em Deep-RL. A navegação dos agentes treinados é comparada com a navegação sem mapa realizada por um algoritmo clássico - BUG2. A abordagem é baseada em dois métodos do estado da arte para navegação sem mapa de robôs terrestres: Política de Gradiente Determinístico Profundo (DDPG) e Soft Actor-Critic (SAC).

show abstract

“…It is also the most trending type of machine learning because it can solve a wide range of complex decisionmaking problems that were previously out of reach. DRL has been applied to the path planning and control problems of mobile robots [24]- [26], UAV [27], and underwater robots [28].…”

Section: Reinforcement Learningmentioning

confidence: 99%

Linear Quadratic Tracking With Reinforcement Learning Based Reference Trajectory Optimization for the Lunar Hopper in Simulated Environment

Tanaka¹,

Cescon

Malki

2021

IEEE Access

View full text Add to dashboard Cite

In this work, we provide a novel optimal guidance and control strategy for lunar hopper obstacle-avoidance, descent, and landing problem and demonstrate its behavior using numerical simulations. More specifically, the major contributions of this paper are three-fold: 1) proposed a feedback-based reference trajectory design for lunar hopper guidance, 2) developed the mathematical models and equations of linear quadratic tracking (LQT) controller for lunar hopper control, and 3) developed a method using reinforcement learning to optimize the designed reference trajectory in conjunction with the designed LQT controller, the so-called linear quadratic tracking with reinforcement learning based reference trajectory optimization (LQT-RTO). We demonstrated the LQT-RTO under a 2-dimensional (2D) lunar hopper simulation environment with 1) the LQT with heuristic reference trajectory design (LQT-HTD) and 2) reinforcement learning (reinforcement learning based controller, or RLC). We confirmed by numerical simulation that the LTQ-RTO outperformed the LQT-HTD in terms of fuel consumption, and outperformed the RLC in terms of landing success rate. Lastly, we provided theoretical interpretation to the simulation results.

show abstract

AUV Position Tracking Control Using End-to-End Deep Reinforcement Learning

Cited by 37 publications

References 14 publications

Deep Reinforcement Learning for End-to-End Local Motion Planning of Autonomous Aerial Robots in Unknown Outdoor Environments: Real-Time Flight Experiments

Deep Reinforcement Learning for End-to-End Local Motion Planning of Autonomous Aerial Robots in Unknown Outdoor Environments: Real-Time Flight Experiments

Aprendizado por Reforço Profundo para Navegação sem Mapa de um Veículo Híbrido Aéreo-Aquático

Linear Quadratic Tracking With Reinforcement Learning Based Reference Trajectory Optimization for the Lunar Hopper in Simulated Environment

Contact Info

Product

Resources

About