A Graph-Based Evolutionary Algorithm: Genetic Network Programming (GNP) and Its Extension Using Reinforcement Learning

Abstract

This paper proposes a graph-based evolutionary algorithm called Genetic Network Programming (GNP). Our goal is to develop GNP, which can deal with dynamic environments efficiently and effectively, based on the distinguished expression ability of the graph (network) structure. The characteristics of GNP are as follows. 1) GNP programs are composed of a number of nodes which execute simple judgment/processing, and these nodes are connected by directed links to each other. 2) The graph structure enables GNP to re-use nodes, thus the structure can be very compact. 3) The node transition of GNP is executed according to its node connections without any terminal nodes, thus the past history of the node transition affects the current node to be used and this characteristic works as an implicit memory function. These structural characteristics are useful for dealing with dynamic environments. Furthermore, we propose an extended algorithm, “GNP with Reinforcement Learning (GNPRL)” which combines evolution and reinforcement learning in order to create effective graph structures and obtain better results in dynamic environments. In this paper, we applied GNP to the problem of determining agents' behavior to evaluate its effectiveness. Tileworld was used as the simulation environment. The results show some advantages for GNP over conventional methods.

Automatic Construction of Programs Using Dynamic Ant Programming

“…In recent years other structures have been investigated in GP (e.g. linear structure (Brameier & Banzhaf, 2001), graph structure (Miller & Smith, 2006;Mabu et al, 2007;Shirakawa & Nagao, 2009)). We plan to apply DAP to construct other structures automatically as well.…”

A Double-Deck Elevator Group Supervisory Control System with Destination Floor Guidance System Using Genetic Network Programming

“…GNP is an extended method of GA and GP (5) (6) . GA has the genome of string structures, while the genome in GP is made of tree structures.…”

“…DFGS is the system which installs the destination call entry system at the hall and adopt some methods to deal with this. This paper proposes EGSCS with DFGS using Genetic Network Programming (GNP) (5) (6) , which is a graph-based evolutionary computation method. Finally, we compare the performances of the system with DFGS and the one without DFGS in DDES through simulations.…”

Genetic network programming with sarsa learning and its application to creating stock trading rules

“…We have proposed Genetic Network Programming (GNP) [1], [2] as an extended method of Genetic Algorithm (GA) [3], [4] and Genetic Programming (GP) [5], [6]. It has been clarified that GNP is an effective method mainly for dynamic problems since GNP represents its solutions using graph structures, which contributes to creating quite compact programs and implicitly memorizing past action sequences in the network flows.…”