Efficient multitask learning with an embodied predictive model for door opening and entry with whole-body control

Abstract

Robots need robust models to effectively perform tasks that humans do on a daily basis. These models often require substantial developmental costs to maintain because they need to be adjusted and adapted over time. Deep reinforcement learning is a powerful approach for acquiring complex real-world models because there is no need for a human to design the model manually. Furthermore, a robot can establish new motions and optimal trajectories that may not have been considered by a human. However, the cost of learning is an issue because it requires a huge amount of trial and error in the real world. Here, we report a method for realizing complicated tasks in the real world with low design and teaching costs based on the principle of prediction error minimization. We devised a module integration method by introducing a mechanism that switches modules based on the prediction error of multiple modules. The robot generates appropriate motions according to the door’s position, color, and pattern with a low teaching cost. We also show that by calculating the prediction error of each module in real time, it is possible to execute a sequence of tasks (opening door outward and passing through) by linking multiple modules and responding to sudden changes in the situation and operating procedures. The experimental results show that the method is effective at enabling a robot to operate autonomously in the real world in response to changes in the environment.

Deep Active Visual Attention for Real-time Robot Motion Generation: Emergence of Tool-body Assimilation and Adaptive Tool-use

“…The inputs are image data i t img and robot joint angle data i t joint at time step t, and the output is the prediction of next joint angles o t+1 joint . The model generates robot motions by repeating the sequence of predicting the next joint angles and applying them to the robot [7], [14], [8].…”

“…By contrast, end-to-end learned models jointly train vision models along with robot control models. This makes the vision models, such as widely used vanilla fully convolutional networks (FCN), to extract task-oriented features even from untrained objects [7], [14], [8]. However, FCNs lack in spatial generalization ability [15], and cannot extract data at untrained visual areas.…”

“…The comparison targets are predicted by a weight shared visual attention module, which is input the actual future image i t+1 img in the dataset. This method is memory efficient compared to previous methods [7], [14], [8], that compute loss of reconstructed images with actual future images. The bi-directional loss not only eliminates redundant image reconstructions, but also induce the visual attention and motion prediction modules to predict attention points that are mutually consistent.…”

“…The proposed model was compared to two existing robot motion prediction models. The models are similarly structured with two modules, but with different vision models: (1) "Vanilla FCN" model [7], [14], which uses vanilla FCNs to extract the feature of the whole image. ( 2) "FCN Attention" model, which directly predicts attention points image feature of FCN output [18], [8].…”

“…Conversely, learning-based methods acquire relationships by learning through interactions of trial and error [2], [3], imitating demonstrated motions [4], [5], or motor babbling [6], which is inspired by human's body babbling behaviours. In particular, end-to-end learning methods [6], [7], [8] directly associated visual and motor sensory data as body schema, which induced vision modules to selectively extract image data that are task-relevant. However, the environment modeled by such vision modules tended to be biased by the training data, which failed to adapt to recognize environment at untrained situations.…”