Genetic Network Programming with Reinforcement Learning and Its Performance Evaluation

Mabu, Shingo; Hirasawa, Kotaro; Hu, Jinglu

doi:10.1007/978-3-540-24855-2_81

Cited by 16 publications

(7 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…GNP represents its solutions as graph structures, which can improve the expression ability and performance. In addition, GNP with Reinforcement Learning (GNP-RL) was proposed a few years ago [29], in [28], GNP with Actor-Critic (GNP-AC) which is a new type of GNP-RL was proposed. Originally, GNP deals with discrete information, but GNP-AC aims to deal with continuous information.…”

Section: B Combinationmentioning

confidence: 99%

A Survey of Approximate Dynamic Programming

Wang

Peng

Zhu

2009

2009 International Conference on Intelligent Human-Machine Systems and Cybernetics

View full text Add to dashboard Cite

Multi-stage decision problems under uncertainty are abundant in process industries. Markov decision process (MDP) is a general mathematical formulation of such problems. Whereas stochastic programming and dynamic programming are the standard methods to solve MDPs, their unwieldy computational requirements limit their usefulness in real applications. Approximate dynamic programming (ADP) combines simulation and function approximation to alleviate the "curse-of-dimensionality" associated with the traditional dynamic programming approach. In this paper, the method of ADP, which abates the curse-of-dimensionality by solving the DP within a carefully chosen, small subset of the state space, was introduced; a survey of recent research directions within the field of ADP had been made.

show abstract

Section: B Combinationmentioning

confidence: 99%

A Survey of Approximate Dynamic Programming

Wang

Peng

Zhu

2009

2009 International Conference on Intelligent Human-Machine Systems and Cybernetics

View full text Add to dashboard Cite

show abstract

“…In the agent circumstances, autonomous agents are required to learn optimal rules for their decision making, so that several learning methods such as reinforcement learning [3,7,21,25,26,28,33] and evolutionary computation [11,16,18,20,22,27,32] have been proposed. In many of these methods, agent behavior is represented as functions from states to actions.…”

Section: Introductionmentioning

confidence: 99%

“…Eq (20). is satisfied, where avg y l (f (y l )) is the function to calculate the average value of f (y l ).…”

mentioning

confidence: 99%

State space segmentation for acquisition of agent behavior

Ueda

Naraki

Nasu

et al. 2008

Web Intelligence and Agent Systems: An International Journal

View full text Add to dashboard Cite

We propose a new method SSED (State Segmentation based on Euclidean Distance) to categorize continuous numeric percepts for Q-learning, where percept vectors are classified into categories and Q-learning uses categories as states to acquire rules for agent behavior. In SSED, categories are represented by hyper-spheres. A percept vector is classified into a category that covers the vector and is the nearest to it. For efficient reinforcement learning, category merging is provided with SSED, where the number of parameters to control category merging in SSED is fewer than that in fuzzy ART with category merging. In addition, match tracking is incorporated into SSED in order to specialize a category. SSED is combined with Q-learning and it is compared with some state segmentation methods. Experimental results show that Q-learning with SSED learns good rules for agent behavior more efficiently than other methods.

show abstract

“…In addition, an online learning algorithm of GNP using Reinforcement Learning (4) (RL) has been proposed (5) (6) and combined with evolution-based GNP (GNP with RL, GNP-RL) (7) (8) . In GNP-RL, since RL is carried out when an agent is carrying out its task, GNP can search for better solutions every judgment and processing during task execution besides the evolutional operation executed after task execution.…”

Section: Introductionmentioning

confidence: 99%

Genetic Network Programming with Reinforcement Learning and Its Application to Making Mobile Robot Behavior

et al. 2006

Self Cite

View full text Add to dashboard Cite

A new graph-based evolutionary algorithm called "Genetic Network Programming (GNP)" has been proposed. GNP represents its solutions as graph structures which have some distinguished abilities. For example, GNP can memorize past action sequences in the network flow and make compact structures. However, conventional GNP is based on evolution, i.e., after GNP programs are carried out to some extent, they are evaluated and evolved based on their fitness values, so many trials must be executed again and again. Therefore, GNP with Reinforcement Learning (GNP-RL) which combines evolution-based GNP and RL has been proposed. In this method, because RL is done when an agent is carrying out its task, GNP can search for better solutions every judgment and processing during task execution besides the evolutional operation executed after task execution. The aim of combining evolution and RL is to take advantage of the sophisticated diversified search ability of evolution and the intensified search ability of RL with online learning. Moreover, the size of the Q table could be set at small values. Therefore, the calculation time and memory consumption can be saved.In this paper, in order to extend the ability of GNP-RL and apply to real world problems, we aim to deal with numerical information (ex. 15 degrees, 512 points, etc.), while all the algorithms of GNP already proposed deal with discrete information (ex. right, left, etc.). In the simulations, the proposed method is applied to the controller of the Khepera simulator and it learns wall-following behavior.Wall-following behavior problem requires the robot to move along walls and also requires it to move as fast and straight as possible. The rewards and fitness values used for learning and evolution, respectively are calculated based on the above conditions. Table 1 shows the generalization ability of GNP-RL, evolution-based GNP (standard GNP) and Neural Networks evolved by GA (NN-GA). The average fitness is calculated as follows. First, each method produces programs in a training Fig. 1. Typical track of the robot controlled by GNP-RL in the testing environment environment. Then, the average fitness is calculated in a testing environment using the best programs of each method obtained in the training environment. From Table 1, it is clarified that GNP-RL shows the best fitness and there are significant differences between GNP-RL and the other methods. Fig. 1 shows a typical behavior of the robot controlled by GNP-RL. It also shows the speed of the wheels under several situations. From the figure, the proposed method can learn wall-following behavior well. The robot can move straight along the wall and when the wall becomes close to the robot, it can appropriately avoid colliding with the wall by adjusting the speed of the two wheels.-11 - Paper Genetic Network Programming with Reinforcement Learning and ItsApplication to Making Mobile Robot Behavior MemberA new graph-based evolutionary algorithm called Genetic Network Programming, GNP has been proposed. The solutions of...

show abstract

Genetic Network Programming with Reinforcement Learning and Its Performance Evaluation

Cited by 16 publications

References 10 publications

A Survey of Approximate Dynamic Programming

A Survey of Approximate Dynamic Programming

State space segmentation for acquisition of agent behavior

Genetic Network Programming with Reinforcement Learning and Its Application to Making Mobile Robot Behavior

Contact Info

Product

Resources

About