Learning Reaching Tasks Using an Arm Robot Equipped with MACMSA

Abstract

In recent years, models that integrate multimodal information to control robots have been actively developed. Memorizing and Associating Converted Multimodal Signal Architecture (MACMSA) was proposed to integrate multimodal information obtained from robots with Hopfield networks as associators and independent feed-forward neural networks as encoders and decoders. The performance of MACMSA has thus far been investigated only using pseudo-data. Notably, MACMSA exhibits high resistance to noise. However, it cannot generate signals for robot control. The purpose of this study was to improve MACMSA to generate signals for robot control and optimize it using real data on reaching tasks. The results of the generated control signals on a real machine are presented to demonstrate that the improved model can be effectively used in a real environment. The results also show that the proposed model can perform well with real data.