Variational Inference for Nonlinear Inverse Problems via Neural Net Kernels: Comparison to Bayesian Neural Networks, Application to Topology Optimization

Abstract

Inverse problems and, in particular, inferring unknown or latent parameters from data are ubiquitous in engineering simulations. A predominant viewpoint in identifying unknown parameters is Bayesian inference where both prior information about the parameters and the information from the observations via likelihood evaluations are incorporated into the inference process. In this paper, we adopt a similar viewpoint with a slightly different numerical procedure from standard inference approaches to provide insight about the localized behavior of unknown underlying parameters. We present a variational inference approach which mainly incorporates the observation data in a point-wise manner, i.e. we invert a limited number of observation data leveraging the gradient information of the forward map with respect to parameters, and find true individual samples of the latent parameters when the forward map is noise-free and one-to-one. For statistical calculations (as the ultimate goal in simulations), a large number of samples are generated from a trained neural network which serves as a transport map from the prior to posterior latent parameters. Our neural network machinery, developed as part of the inference framework and referred to as Neural Net Kernels (NNK), is based on hierarchical (deep) kernels which provide greater flexibility for training compared to standard neural networks. We showcase the effectiveness of our inference procedure in identifying bimodal and irregular distributions compared to a number of approaches including a maximum a posteriori probability (MAP)-based approach, Markov Chain Monte Carlo sampling with both Metropolis-Hastings and Hamiltonian Monte Carlo algorithms, and a Bayesian neural network approach, namely the widely-known Bayes by Backprop algorithm via a pedagogical example. We further apply our inference procedure to two and three dimensional topology optimization problems where we identify the latent parameters in the random field elastic modulus, modeled as a Karhunen-Loéve expansion, considering the high dimensional design iterate-displacement pair as training data. As future research, we discuss the application of this inference approach in identifying constitutive models in nonlinear elasticity and development of fast linear algebraic solvers for large scale similar, i.e. finite element-based inverse problems calculations.