Combining Physical Simulators and Object-Based Networks for Control

Abstract

Physics engines play an important role in robot planning and control; however, many real-world control problems involve complex contact dynamics that cannot be characterized analytically. Most physics engines therefore employ approximations that lead to a loss in precision. In this paper, we propose a hybrid dynamics model, simulator-augmented interaction networks (SAIN), combining a physics engine with an object-based neural network for dynamics modeling. Compared with existing models that are purely analytical or purely data-driven, our hybrid model captures the dynamics of interacting objects in a more accurate and data-efficient manner. Experiments both in simulation and on a real robot suggest that it also leads to better performance when used in complex control tasks. Finally, we show that our model generalizes to novel environments with varying object shapes and materials.

Learning to see the physical world

“…We have developed a hybrid model that captures object-based dynamics by integrating analytical models and neural nets. It assists the robot in accomplishing a highly underactuated task: pushing the right disk to the target (green) by only interacting with the left disk [ Ajay et al, 2019Ajay et al, , 2018. C. D. Particle-based dynamics models support controlling soft robots [Hu et al, 2019] and manipulating deformable objects and liquids [Li et al, 2019b,c].…”

“…These dynamics models can be used in various control tasks: they help to solve highly underactuated control problems (pushing disk A, which in turn pushes disk I B to the target position) [Ajay et al, 2019], to control and co-design soft robots [Hu et al, 2019], to manipulate fluids and rigid bodies on a Kuka robot , and to interact and play games such as Jenga that involve complex frictional micro-interactions .…”

Learning Intuitive Physics and One‐Shot Imitation Using State‐Action‐Prediction Self‐Organizing Maps

“…is meta-learning autoencoder (MeLA) framework is able to build models for previously unseen tasks, which closely match the true underlying models [46]. A simulator-augmented interaction networks (SAIN) model was proposed in [47], which combines object-based networks, which learn residuals, with an object-based learnable physics engine to search for actions in control problems.…”

Physics-Based Neural Network Models for Prediction of Cam-Follower Dynamics Beyond Nominal Operations

“…For instance, enhanced generalization can be obtained by enforcing physical consistency (i.e., conservation of energy) in the loss function [22]. Alternatively, the influence of the neural networks can be attenuated by using them as mappings that compensate for prediction discrepancies of simplified physicsbased models [23], [24]. Furthermore, neural networks have been used to accommodate specific unknown interactions in incomplete yet accurate physics models [25], [26].…”