XCS-based reinforcement learning algorithm for motion planning of a spherical mobile robot

Roozegar, M.; Mahjoob, M. J.; Esfandyari, Mohammad Javad; Panahi, Masoud Shariat

doi:10.1007/s10489-016-0788-9

Cited by 23 publications

(12 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Once the free circle area is decided, the sphere can move forward or backward directly to make sure the left leaning can reach the edge of free space and move left forward a certain distance. As shown in Figure 10, a left forward motion is illustrated, and relevant parameter is shown in equation (8). The equation for f ð'Þ is shown in equation ( 9), and this equation shares the same monotonicity with the function yð'Þ ¼ q when ' 2 ½0; ' max Optimal parameter diagram of optimal tilt angle ' and expanded rad q.…”

Section: Minimal Area For Rotationmentioning

confidence: 93%

“…2 Roozegar also proposed a model-free reinforcement learning algorithm based on extended classifier system to navigate a spherical robot, which is an end-to-end method of both plan and control for spherical robot. 8 The purpose of navigation for a robot is to find a physically achievable collision-free path to guide the robot to the target, avoiding all the obstacles at the same time, which guarantees that the generated path can be kept in line by the specific nonholonomic robot with a specific model, considering its kinematic constraints and dynamic constraints. A lot of research have been done on car-like robots.…”

Section: Introductionmentioning

confidence: 99%

“…Figure16. Three generated paths (7),(8), and (9) in complicated maze scenario: (a) Path (7), (b) path(8), and (c) path(9).…”

mentioning

confidence: 99%

See 2 more Smart Citations

Improved hybrid A* path planning method for spherical mobile robot based on pendulum

Zhang

Wan

You

et al. 2021

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

This article proposes a modification of hybrid A* method used for navigation of spherical mobile robots with the ability of limited partial lateral movement driven by pendulum. For pendulum-driven spherical robots with nonzero minimal turning radius, our modification helps to find a feasible and achievable path, which can be followed in line with the low time cost. Because of spherical shell shape, the robot is point contact with the ground, showing different kinematic model compared with common ground mobile robots such as differential robot and wheeled car-like robot. Therefore, this article analyzes the kinematic model of spherical robot and proposes a novel method to generate feasible and achievable paths conforming to kinematic constraints, which can be the initial value of future trajectory tracking control and further optimization. A concept of optimal robot’s minimum area for rotation is also proposed to improve search efficiency and ensure the ability of turning to any orientation by moving forward and backward in a finite number of times within limited areas.

show abstract

Section: Minimal Area For Rotationmentioning

confidence: 93%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Improved hybrid A* path planning method for spherical mobile robot based on pendulum

Zhang

Wan

You

et al. 2021

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

show abstract

“…Optimal motion planning and control of a spherical mobile robot using Bellman's dynamic programming methodology was recently developed [32–34]. As well, based on reinforcement learning algorithm and the notion of learning agents, a direct approach to motion planning of a spherical robot was proposed [35, 36]. A novel omnidirectional spherical robot, with a driven ball installed inside the spherical shell, was designed and implemented, too [37].…”

Section: Introductionmentioning

confidence: 99%

Modelling and control of a non‐holonomic pendulum‐driven spherical robot moving on an inclined plane: simulation and experimental results

Roozegar

Mahjoob

2017

IET Control Theory & Applications

View full text Add to dashboard Cite

“…So a "teacher" is unavailable. In the learning process, XCS adopts the reinforcement learning approach, which will allow the robot to take actions in the environment and learn solutions to improve its performance [71,72]. The robot's training process makes XCS more suitable for this robotic application than UCS (sUpervised Classifier System) [73,74], which is designed for supervised problems where the "correct" actions are available in the training process [75].…”

Section: Learning Classifier Systemsmentioning

confidence: 99%

An Affective Computing Multilayer Cognitive Architecture for Evolutionary Cognitive Robotics

Zhang¹

View full text Add to dashboard Cite

<p>Robots are entering our daily lives from self-driving cars to health-care robots. Historically, pre-programmed robots were vulnerable to changing conditions in daily life, primarily because of a lack of ability to generate novel, non-preset flexible solutions. Thus there is a need for robotics to incorporate adaptation, which is a trait of higher order natural species. This adaptation allows higher-order natural species to change their behaviours and internal mechanisms based on experience with often dynamic environment. The ability to adapt emerged through evolutionary processes. Evolutionary Robotics is an approach to create autonomous robots that are capable of automatically generating artificial behaviors and morphologies to achieve adaptation. Evolutionary robotics has the potential to automatically synthesize controllers for real autonomous robots and generate solutions to complete tasks in the uncertain real-world. Compared to the inflexibility of pre-programmed robots, evolutionary robots are able to learn flexible solutions to given tasks through evolutionary methods. Cognitive robotics, a branch of artificial cognitive systems research, is such an attempt to create autonomous robots by applying bio-inspired methods. As the robot interacts with environment, an underlying cognitive system can learn its own solutions toward task completion. This learning-solution-from-interaction approach, also termed as a Reinforcement Learning (RL) approach, is widely applied in cognitive robotics to learn the solutions automatically. Ideally, the solutions can emerge in the cognitive system through the trial-and-error process of the RL approach without introducing human bias. This thesis aims to develop an evolutionary cognitive architecture (system) for a robot that can learn adaptive solutions to complete tasks. Inspired by emotion theories, this work proposes Affective Computing Multilayer Cognitive Architecture (ACMCA), a universal cognitive architecture, which is able to learn diverse solutions. Extending from previous work, ACMCA has a five-layer structure, where each layer aims to achieve different components of the solutions. The position of this thesis is that introducing a novel emotion inspired multilayer architecture that produces task solutions through subsumption operations and underlying appropriate machine learning algorithms will allow a robot to complete admissible tasks. ACMCA’s five layers are: primary reinforcer layer, secondary reinforcer layer, core affect state layer, strategy layer, and behaviour layer. This five-layer decomposition also meets the traditional decomposition of a mobile control system into functional modules (e.g. perception, modelling, planning, task execution, and motor control). Each layer contains computing nodes as functional modules that process various Stimuli, Actions, and their consequential Outcomes of the cognitive system. In this work, 17 computing nodes and their connections in ACMCA represent the solutions that a mobile robot has learned to complete navigation tasks in complex scenarios. Inspired by the Constructive Theory 1 and the robotic subsumption system, this work proposes a contingency-based subsumption approach to construct ACMCA. This contingency is termed Stimuli-Action-Outcome Contingency (SAOC), which is extended from the Action-Outcome (AO) contingency of Construction Theory. SAOCs are represented by “if-then” rules, termed SAOC rules, which encapsulate Stimuli, Actions, and their consequential Outcomes, providing clear symbolic interpretations. That is, the symbolic meaning of a SAOC rule can be interpreted as: if the input stimulus is perceived, the output action will be advocated as a cognitive response, expecting the outcome of the action with an estimation of relevance. As low-level computing nodes encapsulate Stimulus, Actions, and Outcomes, high-level computing nodes can subsume these low-level ones through the form of SAOC rules. Therefore, the proposed ACMCA can be constructed by subsumption layers of Stimuli-Action-Outcome Contingency (SAOC) rules. This work applies machine learning techniques to facilitate ACMCA’s real-world robotic implementation. This work selects Accuracy-based Learning Classifier Systems (XCS) algorithms as the underlying machine learning techniques that are deployed at computing nodes for the contingency-based subsumption operations. The mitosis approach of XCS and the XCS with a Combined Reward method (XCSCR) are two novel variants of XCS algorithm. They are proposed to amend two challenges that occur when the standard XCS approaches are applied for robotic applications. The mitosis approach introduces an accuracy pressure into the algorithm’s evolutionary process, improving the algorithms’ performance in robotic applications where noisy interferences exist. The XCSCR enables the policy to emerge earlier and more frequently than the existing benchmark approaches in multistep problems. Therefore, a robot with the XCSCR can handle a multistep scenario more effectively than those with the benchmarked algorithms. This work conducts five experiments to test the capability of ACMCA and its underlying algorithms in learning solutions for robotic navigation tasks. The five experiments are conducted as follows: reflex-learning, IR-tuning, deliberation-establishing, emotion model, and combined reward assignment. As the results of the experiments, three different affective patterns have emerged in the first three experiments, an emotion model has emerged in the fourth experiments, and the fifth experiment explores ACMCA’s potential implementation in the life-long learning scenario. These results demonstrate that ACMCA, a novel emotion inspired multilayer architecture, can produce task solutions through contingency-based subsumption operations and underlying appropriate machine learning algorithms, allowing a robot to complete admissible tasks through evolutionary processes. The contingency-based subsumption operations can establish three contingencies and one emotion model between the subsumed components by multiple RL agents which deploy the proposed mitosis approach of XCS algorithms. These three emotion patterns and emotion model can consistently improve the robot’s navigation performance with interpretable explanations. These two variants of XCS algorithms can amend shortfalls of the standard XCS approach in real-world robotic implementations. It has been demonstrated that the diverse solutions learned by ACMCA improve the navigation performance of the robot in terms of higher flexibility, reduction in continuous collisions and shorter navigation time consumption.</p>

show abstract

XCS-based reinforcement learning algorithm for motion planning of a spherical mobile robot

Cited by 23 publications

References 16 publications

Improved hybrid A* path planning method for spherical mobile robot based on pendulum

Improved hybrid A* path planning method for spherical mobile robot based on pendulum

Modelling and control of a non‐holonomic pendulum‐driven spherical robot moving on an inclined plane: simulation and experimental results

An Affective Computing Multilayer Cognitive Architecture for Evolutionary Cognitive Robotics

Contact Info

Product

Resources

About