Elevator Group Supervisory Control System Using Genetic Network Programming with Macro Nodes and Reinforcement Learning

Abstract

Keywords: elevator group supervisory control system, genetic network programming, reinforcement learning, importance weight GA evolves strings and it is mainly applied to optimization problems, and GP was devised later in order to expand the expression ability of GA by using tree structures. This structural change of solutions brought progress on the evolutionary computation and made GP applicable to more complex problems. Since the past studies suggested that different structure has different expression ability, a new evolutionary computation method called Genetic Network Programming (GNP) with directed network structures has been proposed. To verify its applicability and efficiency, some studies have been done on both virtual and real world problems.GNP was firstly applied to Elevator Group Supervisory Control System (EGSCS) in a real world problem after its applicability and efficiency has been clarified in virtual world such as tile-world model and ant colonies, etc. Simulation test results demonstrated that GNP could also work well on such a complex stochastic optimal control problem. However, even though some improvements of the EGSCS' performances over the conventional control methods have been made using GNP, there remain some problems such as searching for faster training speed and better performances. In EGSCS using only GNP, the GNP structure and its parameters are optimized by genetic operations at the end of each generation. That is to say, there is nothing optimized during the individual execution process. On the other hand, an extended algorithm of GNP combining learning and evolution called Genetic Network Programming with Reinforcement Learning (GNP with RL) has been proposed and its efficiency has been also verified with some benchmark problems. Comparing to the benchmark problems such as tileworld, EGSCS is a more complex real world problem. In this paper, we should study an appropriate algorithm based on the inherent nature of EGSCS because the method using GNP with RL in benchmark problems can not be employed in EGSCS directly and easily. Therefore, in this paper, we Fig. 1. Basic Structure of an EGSCS using GNP with RL propose a new method for EGSCS using GNP with Macro Nodes and Reinforcement Learning (GNP-RL) as shown in Fig.1.With only the GNP system, evolution takes place after a set of simulation runs are completed and the fitness of each individual GNP is evaluated. In contrast with this, when RL is added, the system could perform online learning even during task execution. Thus, we could expect faster learning and improved performances. The performance of the proposed method is studied by simulations under several conditions. Some analyses are made based on these test results comparing to other algorithms using original GNP and conventional control methods. Moreover, to explore further the efficiency of the proposed method, we optimize the framework of GNP with RL by tuning the importance weight of the macro-processing node. And the experiment results show that some better performanc...

A nonlinear model to rank association rules based on semantic similarity and genetic network programing

“…In recent years, there has been a variety of successful applications of GNP, including Data Mining [5][6][7][8], Intelligent Mobile Robot [16], Multi Agent System [18], Elevator Group Supervisory Control System [20,21], Stock Market Prediction [22,23], Automatic Program Generation [24], and so on.…”