Obstacle Avoidance Using Stereo Vision and Deep Reinforcement Learning in an Animal-like Robot

Ling, Fuhai; Jiménez-Rodríguez, Alejandro; Prescott, Tony J.

doi:10.1109/robio49542.2019.8961639

Cited by 2 publications

(4 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly, a robot may have a separate local navigation module that would generate a course diversion to avoid hitting objects. We have previously demonstrated how this local navigation could be acquired through RL [35,48]. More generally, control architectures in both animals and robots are likely to distinguish between the global and local navigation problems and employ separate mechanisms for each [44].…”

Section: Discussionmentioning

confidence: 99%

“…The MiRo robot is a commercially available biomimetic robot developed by Consequential Robotics Ltd in partnership with the University of Sheffield. MiRo's physical design and control system architecture find their inspiration in biology, psychology and neuroscience [39], making it a valuable platform for embedded testing of brain-inspired models of perception, memory and learning [35]. For mobility, the robot is differentially driven, whilst we use its front-facing sonar to detect approaching walls and objects for sensing.…”

Section: Miro Robot and The Testing Environmentmentioning

confidence: 99%

See 1 more Smart Citation

A robotic model of hippocampal reverse replay for reinforcement learning

Whelan¹,

Prescott²,

Vasilaki

2022

Bioinspir. Biomim.

View full text Add to dashboard Cite

Hippocampal reverse replay is thought to contribute to learning, and particularly reinforcement learning, in animals. We present a computational model of learning in the hippocampus that builds on a previous model of the hippocampal-striatal network viewed as implementing a three-factor reinforcement learning rule. To augment this model with hippocampal reverse replay, a novel policy gradient learning rule is derived that associates place cell activity with responses in cells representing actions. This new model is evaluated using a simulated robot spatial navigation task inspired by the Morris water maze. Results show that reverse replay can accelerate learning from reinforcement, whilst improving stability and robustness over multiple trials. As implied by the neurobiological data, our study implies that reverse replay can make a significant positive contribution to reinforcement learning, although learning that is less efficient and less stable is possible in its absence. We conclude that reverse replay may enhance reinforcement learning in the mammalian hippocampal-striatal system rather than provide its core mechanism.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Miro Robot and The Testing Environmentmentioning

confidence: 99%

A robotic model of hippocampal reverse replay for reinforcement learning

Whelan¹,

Prescott²,

Vasilaki

2022

Bioinspir. Biomim.

View full text Add to dashboard Cite

show abstract

“…By employing reinforcement learning, particularly the Dyna-Q approach, quadcopters can enhance their decision-making and adapt their flight trajectories. This combination of strategic path planning and adaptive obstacle avoidance, aided by advanced machine learning, allows quadcopters to optimize their operations, prevent collisions, and maintain stability while dynamically adjusting to their surroundings and achieving mission objectives [27][28][29][30][31].…”

Section: Enhancing Quadcopter Trajectory Tracking Through Dyna-q Lear...mentioning

confidence: 99%

“…Dyna-Q learning combines both learning the model and Q-learning to optimize the learning process effectively. In reinforcement learning, the Markov Decision Process (MDP) is used to model the interactions between an agent and the environment, helping the agent maximize cumulative rewards in uncertain environments [12,27,29,32,33]. MDPs aim to determine policies that guide the agent's actions:…”

Section: Agent Environmentmentioning

confidence: 99%

Enhancing Quadcopter Autonomy: Implementing Advanced Control Strategies and Intelligent Trajectory Planning

Hadid,

Boushaki,

Boumchedda

et al. 2024

Automation

View full text Add to dashboard Cite

In this work, an in-depth investigation into enhancing quadcopter autonomy and control capabilities is presented. The focus lies on the development and implementation of three conventional control strategies to regulate the behavior of quadcopter UAVs: a proportional–integral–derivative (PID) controller, a sliding mode controller, and a fractional-order PID (FOPID) controller. Utilizing careful adjustments and fine-tuning, each control strategy is customized to attain the desired dynamic response and stability during quadcopter flight. Additionally, an approach called Dyna-Q learning for obstacle avoidance is introduced and seamlessly integrated into the control system. Leveraging MATLAB as a powerful tool, the quadcopter is empowered to autonomously navigate complex environments, adeptly avoiding obstacles through real-time learning and decision-making processes. Extensive simulation experiments and evaluations, conducted in MATLAB 2018a, precisely compare the performance of the different control strategies, including the Dyna-Q learning-based obstacle avoidance technique. This comprehensive analysis allows us to understand the strengths and limitations of each approach, guiding the selection of the most effective control strategy for specific application scenarios. Overall, this research presents valuable insights and solutions for optimizing flight stability and enabling secure and efficient operations in diverse real-world scenarios.

show abstract

Obstacle Avoidance Using Stereo Vision and Deep Reinforcement Learning in an Animal-like Robot

Abstract: This is a repository copy of Obstacle avoidance using stereo vision and deep reinforcement learning in an animal-like robot.

Cited by 2 publications

References 16 publications

A robotic model of hippocampal reverse replay for reinforcement learning

A robotic model of hippocampal reverse replay for reinforcement learning

Enhancing Quadcopter Autonomy: Implementing Advanced Control Strategies and Intelligent Trajectory Planning

Contact Info

Product

Resources

About