This paper presents some results from a study of biped dynamic walking using reinforcement learning. During this study a hardware biped robot was built, a new reinforcement learning algorithm as well as a new learning architecture were developed. The biped learned dynamic walking without any previous knowledge about its dynamic model. The Self Scaling Reinforcement learning algorithm was developed in order to deal with the problem of reinforcement learning in continuous action domains. The learning architecture was developed in order to solve complex control problems. It uses different modules that consist of simple controllers and small neural networks. The architecture allows for easy incorporation of new modules that represent new knowledge, or new requirements for the desired task.

No abstract

S performance. In this work we focus on two skills, namely robotic assembly and balancing and on two classic tasks to develop these skills via Ieaming: the peg-in-hole insertion task, and the ball balancing task. A stochastic real-valued (SRV) reinforcement leaming algorithm is described and used for leam-is with the Conipiiter Sc,ienc,e Depurtnietit, Utii\wsity of Massachusetts. Amhcr..>t. MA 01003. Enruil: ,.ijuyku-mar@cs.umass.edu. J.A. Fixnkliii und H . Benhrulrinr are Mith GTE Labor-utories Incorporated, 40 S y l~m Roud. Wulthuni. MA 02254. Email: ,$runkliri@gte.cwn and hhc~rihruhini~,~tc..c.om. The w w k of V: Gullapalli was siippor.ted by firriding to A. Barto by the AFOSR.Skilled behavior involves the effective use of knowledge in execution or performance. A skill may require dexterity or coordination, and generally develops over time through leaming. This work focuses on employing leaming to enable a robot to acquire skills, particularly physical skills where leaming control is required. The requirements are i) dealing with nonlinearity/complex dynamics, ii) achieving robust performance under