Materializing Architecture for Processing Multimodal Signals for a Humanoid Robot Control System

Abstract

In recent years, many systems have been developed to embed deep learning in robots. Some use multimodal information to achieve higher accuracy. In this paper, we highlight three aspects of such systems: cost, robustness, and system optimization. First, because the optimization of large architectures using real environments is computationally expensive, developing such architectures is difficult. Second, in a real-world environment, noise, such as changes in lighting, is often contained in the input. Thus, the architecture should be robust against noise. Finally, it can be difficult to coordinate a system composed of individually optimized modules; thus, the system is better optimized as one architecture. To address these aspects, a simple and highly robust architecture, namely memorizing and associating converted multimodal signal architecture (MACMSA), is proposed in this study. Verification experiments are conducted, and the potential of the proposed architecture is discussed. The experimental results show that MACMSA diminishes the effects of noise and obtains substantially higher robustness than a simple autoencoder. MACMSA takes us one step closer to building robots that can truly interact with humans.

Learning Reaching Tasks Using an Arm Robot Equipped with MACMSA

“…The second three layers were used as the decoders. In other words, the encoder and decoder shared a single hidden layer [11]. Note that, in the current study, although all the modes of the encoders and decoders were optimized as autoencoders, and the decoder was used for the actuator positions.…”

“…The optimizer used was Adam [14]. Notably, the loss function is the sum of squares added to the Kullbacker divergence (Equation ( 2)) [11].…”

“…Some robot control systems have also been developed to process multimodal information to obtain higher recognition [6][7][8][9][10]. Systems that process multimodal information face two challenges [11]. The first is the increase in the computational cost.…”

“…Memorizing and Associating Converted Multimodal Signal Architecture (MACMSA) [11], which is used for integrating multimodal information obtained from robots by adopting the Hopfield network as an associator and independent feed-forward neural networks as encoders and decoders, was proposed to resolve the two challenges. MACMSA uses independent encoders for each modal, which contributes to reducing network bloats even when the number of modals increases.…”

“…First, the performance of the model was evaluated using only pseudo-data, not real data. In [11], the pseudo-data that were generated had a normal distribution. However, real data obtained from robots are not always normally distributed.…”