Augmenting Physical Simulators with Stochastic Neural Networks: Case Study of Planar Pushing and Bouncing

Abstract

An efficient, generalizable physical simulator with universal uncertainty estimates has wide applications in robot state estimation, planning, and control. In this paper, we build such a simulator for two scenarios, planar pushing and ball bouncing, by augmenting an analytical rigid-body simulator with a neural network that learns to model uncertainty as residuals. Combining symbolic, deterministic simulators with learnable, stochastic neural nets provides us with expressiveness, efficiency, and generalizability simultaneously. Our model outperforms both purely analytical and purely learned simulators consistently on real, standard benchmarks. Compared with methods that model uncertainty using Gaussian processes, our model runs much faster, generalizes better to new object shapes, and is able to characterize the complex distribution of object trajectories.

TossingBot: Learning to Throw Arbitrary Objects with Residual Physics

“…This leads to a more restricted set of grasps in contrast to column 4, where the throwing can learn to compensate respectively. Across all policies, the histograms visualizing grasps which lead to successful throws (columns 2, 5, 8) share large overlaps with the grasps that lead to failed throws (red columns 3,6,9). This suggests grasping and throwing might have been learned simultaneously, rather than one after the other -likely because the way the robot throws is not trivially conditioned on how it grasps.…”

Omnipush: accurate, diverse, real-world dataset of pushing dynamics with RGB-D video

“…Since we presented our earlier dataset on planar pushing [1], it has been directly used for: 1) Stochastic modeling: [23,10,17] 2) Modeling from rendered images: [16] 3) Model identification: [24] 4) Learning models for control: [25,26] 5) Filtering: [27] 6) Meta-learning: [28] With this new dataset we hope to further facilitate research in learning models and control.…”

Combining Physical Simulators and Object-Based Networks for Control

“…This is the intuition behind the observation that residual learning allows easier generalization to novel scenarios. Ajay [9] validated this for forward prediction. Here, we evaluate how our fine-tuned SAIN generalizes for control.…”

“…The paper closest to ours is that from Ajay et al [9], where they used the analytical model as an approximation to the push outcomes, and learned a residual neural model that makes corrections to its output. In contrast, our paper makes two key innovations: first, instead of using a feedforward network to model the dynamics of a single object, we employ an objectbased network to learn residuals.…”

“…easier and more sample efficient than learning from scratch. This approach has been shown to be more data efficient, have better generalization capabilities, and outperform its purely analytical or data-driven counterparts [8], [9], [10], [11]. Most residual-based approaches assume a fixed number of objects in the world states.…”