Optimizing Hyperparameters and Architecture of Deep Energy Method

Abstract

The deep energy method (DEM) employs the principle of minimum potential energy to train neural network models to predict displacement at a state of equilibrium under given boundary conditions. The accuracy of the model is contingent upon choosing appropriate hyperparameters. The hyperparameters have traditionally been chosen based on literature or through manual iterations. The displacements predicted using hyperparameters suggested in the literature do not ensure the minimum potential energy of the system. Additionally, they do not necessarily generalize to different load cases. Selecting hyperparameters through manual trial and error and grid search algorithms can be highly time-consuming. We propose a systematic approach using the Bayesian optimization algorithms and random search to identify optimal values for these parameters. Seven hyperparameters are optimized to obtain the minimum potential energy of the system under compression, tension, and bending loads cases. In addition to Bayesian optimization, Fourier feature mapping is also introduced to improve accuracy. The models trained using optimal hyperparameters and Fourier feature mapping could accurately predict deflections compared to finite element analysis for linear elastic materials. The deflections obtained for tension and compression load cases are found to be more sensitive to values of hyperparameters compared to bending. The approach can be easily extended to 3D and other material models.

On the use of graph neural networks and shape‐function‐based gradient computation in the deep energy method

“…The studies by Chadha et al 7 and He et al 34 demonstrated gradient computation and numerical integration through finite element SFs and Gauss quadrature. In this case, the spatial gradients are given by: $$$$ \frac{\partial \boldsymbol{u}\left(\boldsymbol{X}\right)}{\partial \boldsymbol{X}}\approx {\boldsymbol{J}}^{-1}\frac{\partial \boldsymbol{\phi}}{\partial \boldsymbol{\xi}}\cdotp \boldsymbol{u}, $$$$ where $$$ \boldsymbol{\phi} $$$, $$$ \boldsymbol{\xi} $$$, and $$$ \boldsymbol{u} $$$ denote the finite element SFs, natural coordinates, and displacement vector (evaluated at discrete nodes), respectively.…”

“…The studies by Chadha et al 7 and He et al 34 demonstrated gradient computation and numerical integration through finite element SFs and Gauss quadrature. In this case, the spatial gradients are given by:…”

“…While shown to be highly effective, it is of interest to study how the underlying neural network (NN) structure affects the performance of the resulting DEM model. Chadha et al 7 studied the effects of changing network architecture (number of neurons and hidden layers) using an MLP‐based DEM model. Zhuang et al 8 investigated the use of autoencoders in the DEM method.…”

“…Therefore, as an alternative, spatial gradients of the field variables can be calculated on a mesh‐based discretization of the domain using Sobel filters (finite difference) 30‐33 . In addition, finite element shape functions (SFs) can be used 7,11,34,35 . In this case, the gradients are evaluated at the integration points of the finite elements through the shape function gradients, identical to the treatment in classical FEM 36,37 .…”

“…[30][31][32][33] In addition, finite element shape functions (SFs) can be used. 7,11,34,35 In this case, the gradients are evaluated at the integration points of the finite elements through the shape function gradients, identical to the treatment in classical FEM. 36,37 We note that a thorough comparison between the AD-and SF-based gradient computations and their implications on the accuracy and stability of the DEM model has been missing from the literature.…”