gradSim: Differentiable simulation for system identification and visuomotor control

Jatavallabhula, Krishna Murthy; Macklin, Miles; Golemo, Florian; Voleti, Vikram; Petrini, Linda; Weiss, Martin; Considine, Breandan; Parent-Levesque, Jerome; Xie, Kevin; Erleben, Kenny; Paull, Liam; Shkurti, Florian; Nowrouzezahrai, Derek; Fidler, Sanja

doi:10.48550/arxiv.2104.02646

Cited by 3 publications

(3 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…input parameters. Whereas some works rely on auto-differentiation [Hu et al 2019a,b;Jatavallabhula et al 2021], others employ the adjoint method to compute the analytical derivative [Chen et al 2021;Geilinger et al 2020;Liang et al 2019;Panetta et al 2019Panetta et al , 2017Panetta et al , 2015Tozoni et al 2021]. Panetta et al [2017; optimize for structure parameters using analytical shape derivatives obtained from a level set formulation.…”

Section: Related Workmentioning

confidence: 99%

Neural Metamaterial Networks for Nonlinear Material Design

Li,

Coros,

Thomaszewski

2023

ACM Trans. Graph.

View full text Add to dashboard Cite

Nonlinear metamaterials with tailored mechanical properties have applications in engineering, medicine, robotics, and beyond. While modeling their macromechanical behavior is challenging in itself, finding structure parameters that lead to ideal approximation of high-level performance goals is a challenging task. In this work, we propose Neural Metamaterial Networks (NMN)---smooth neural representations that encode the nonlinear mechanics of entire metamaterial families. Given structure parameters as input, NMN return continuously differentiable strain energy density functions, thus guaranteeing conservative forces by construction. Though trained on simulation data, NMN do not inherit the discontinuities resulting from topo-logical changes in finite element meshes. They instead provide a smooth map from parameter to performance space that is fully differentiable and thus well-suited for gradient-based optimization. On this basis, we formulate inverse material design as a nonlinear programming problem that leverages neural networks for both objective functions and constraints. We use this approach to automatically design materials with desired strain-stress curves, prescribed directional stiffness and Poisson ratio profiles. We furthermore conduct ablation studies on network nonlinearities and show the advantages of our approach compared to native-scale optimization.

show abstract

Section: Related Workmentioning

confidence: 99%

Neural Metamaterial Networks for Nonlinear Material Design

Li,

Coros,

Thomaszewski

2023

ACM Trans. Graph.

View full text Add to dashboard Cite

show abstract

“…The early works focused initially on simple rigid bodies (de Avila Belbute-Peres et al, 2018a;Degrave et al, 2019) and later simulation of high degrees of freedom systems, such as fluids (Schenck & Fox, 2018), elastic bodies (Hu et al, 2019;Huang et al, 2021), and cloth (Liang et al, 2019). More recently, Jatavallabhula et al (2021) introduced an end-to-end differentiable simulator that can learn from images by combining differentiable rendering and differentiable simulation. Comparatively, we explore fine-grained DPMs for composite materials, which leads to new challenges in differentiable modeling.…”

Section: Related Workmentioning

confidence: 99%

Fine-grained differentiable physics: a yarn-level model for fabrics

Gong¹,

Zhu²,

Bulpitt³

et al. 2022

Preprint

View full text Add to dashboard Cite

Differentiable physics modeling combines physics models with gradient-based learning to provide model explicability and data efficiency. It has been used to learn dynamics, solve inverse problems and facilitate design, and is at its inception of impact. Current successes have concentrated on general physics models such as rigid bodies, deformable sheets, etc, assuming relatively simple structures and forces. Their granularity is intrinsically coarse and therefore incapable of modelling complex physical phenomena. Fine-grained models are still to be developed to incorporate sophisticated material structures and force interactions with gradient-based learning. Following this motivation, we propose a new differentiable fabrics model for composite materials such as cloths, where we dive into the granularity of yarns and model individual yarn physics and yarn-to-yarn interactions. To this end, we propose several differentiable forces, whose counterparts in empirical physics are indifferentiable, to facilitate gradient-based learning. These forces, albeit applied to cloths, are ubiquitous in various physical systems. Through comprehensive evaluation and comparison, we demonstrate our model's explicability in learning meaningful physical parameters, versatility in incorporating complex physical structures and heterogeneous materials, data-efficiency in learning, and high-fidelity in capturing subtle dynamics. Code is available in: https://github.com/realcrane/Fine-grained-Differentiable-P hysics-A-Yarn-level-Model-for-Fabrics.git

show abstract

“…However, these works sacrifice generality and accuracy of physics for differentiability of the neural network. Differentiable simulators have been developed for rigid bodies [3,11,12,16,25,52], soft bodies [15,22,26,27,28,30], and cloth [33,42]. Our simulator lies in the rigid body category, with some key modifications and improvements that enable contact-rich tasks through differentiability of frictional contact between multiple dynamic bodies.…”

Section: A Differentiable Physics-based Simulationmentioning

confidence: 99%

An End-to-End Differentiable Framework for Contact-Aware Robot Design

Xu,

Chen,

Zlokapa

et al. 2021

Preprint

View full text Add to dashboard Cite

The current dominant paradigm for robotic manipulation involves two separate stages: manipulator design and control. Because the robot's morphology and how it can be controlled are intimately linked, joint optimization of design and control can significantly improve performance. Existing methods for co-optimization are limited and fail to explore a rich space of designs. The primary reason is the trade-off between the complexity of designs that is necessary for contact-rich tasks against the practical constraints of manufacturing, optimization, contact handling, etc. We overcome several of these challenges by building an end-to-end differentiable framework for contactaware robot design. The two key components of this framework are: a novel deformation-based parameterization that allows for the design of articulated rigid robots with arbitrary, complex geometry, and a differentiable rigid body simulator that can handle contact-rich scenarios and computes analytical gradients for a full spectrum of kinematic and dynamic parameters. On multiple manipulation tasks, our framework outperforms existing methods that either only optimize for control or for design using alternate representations or co-optimize using gradient-free methods.

show abstract

gradSim: Differentiable simulation for system identification and visuomotor control

Abstract: Figure 1: ∇Sim is a unified differentiable rendering and multiphysics framework that allows solving a range of control and parameter estimation tasks (rigid bodies, deformable solids, and cloth) directly from images/video.

Cited by 3 publications

References 43 publications

Neural Metamaterial Networks for Nonlinear Material Design

Neural Metamaterial Networks for Nonlinear Material Design

Fine-grained differentiable physics: a yarn-level model for fabrics

An End-to-End Differentiable Framework for Contact-Aware Robot Design

Contact Info

Product

Resources

About