Extracting Interpretable Physical Parameters from Spatiotemporal Systems Using Unsupervised Learning

Abstract

Experimental data are often affected by uncontrolled variables that make analysis and interpretation difficult. For spatiotemporal systems, this problem is further exacerbated by their intricate dynamics. Modern machine learning methods are particularly well suited for analyzing and modeling complex datasets, but to be effective in science, the result needs to be interpretable. We demonstrate an unsupervised learning technique for extracting interpretable physical parameters from noisy spatiotemporal data and for building a transferable model of the system. In particular, we implement a physics-informed architecture based on variational autoencoders that is designed for analyzing systems governed by partial differential equations. The architecture is trained end to end and extracts latent parameters that parametrize the dynamics of a learned predictive model for the system. To test our method, we train our model on simulated data from a variety of partial differential equations with varying dynamical parameters that act as uncontrolled variables. Numerical experiments show that our method can accurately identify relevant parameters and extract them from raw and even noisy spatiotemporal data (tested with roughly 10% added noise). These extracted parameters correlate well (linearly with R 2 > 0.95) with the ground truth physical parameters used to generate the datasets. We then apply this method to nonlinear fiber propagation data, generated by an ab initio simulation, to demonstrate its capabilities on a more realistic dataset. Our method for discovering interpretable latent parameters in spatiotemporal systems will allow us to better analyze and understand real-world phenomena and datasets, which often have unknown and uncontrolled variables that alter the system dynamics and cause varying behaviors that are difficult to disentangle.

Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discovery

“…For example, Zheng et al [11] used multilayer perceptrons to extract relevant properties from a system of bouncing balls (such as the mass of the balls or the spring constant of a force between the balls) and simultaneously predicted the trajectory of a different set of objects. Lu et al [12] accomplished a similar goal but using a dynamics encoder (DE) with convolutional layers and a propagating decoder (PD) with deconvolutional layers Here, we present a deep learning architecture shown in Fig. 4, which is based on the DE-PD architecture.…”

“…The DE takes in the full input series {x t } T x t=0 over T x time steps and outputs a single-dimensional latent variable z. Unlike the original DE-PD architecture presented in [12], the DE here is not a VAE. The DE here consists of several convolutional layers followed by fully connected layers and a batch normalization layer.…”

“…Trask et al [10] proposed a neural network module called the neural arithmetic logic unit (NALU) that introduces inductive biases toward arithmetic operations so that the architecture can extrapolate well on specific tasks. Neural network-based architectures have also been used to extract relevant and interpretable parameters from dynamical systems and use these parameters to predict the propagation of a similar system [11], [12]. In addition, Chari et al [13] used symbolic regression as a separate module to discover kinematic equations using parameters extracted from videos regarding various types of motion.…”

Interpretable and Explainable Machine Learning for Materials Science and Chemistry

“…As another example, the so-called β-VAE [62] introduces additional constraints to enforce orthogonality and sparsity on the latent space, so that the dimensions are uncorrelated and the VAE will only use the minimum number of dimensions required for reconstruction of the data. This kind of β-VAE type approach was recently shown to extract parameters that are interpretable as the driving parameters of ordinary differential equations from data of dynamic processes [63].…”

Surrogate- and invariance-boosted contrastive learning for data-scarce applications in science

“…In recent years, there has been increasing interest and rapid advances in applying data-driven approaches, in particular deep learning via neural networks, to problems in the natural sciences [1][2][3][4]. Unlike traditional physics-informed approaches, deep learning relies on extensive amounts of data to quantitatively discover hidden patterns and correlations to perform tasks such as predictive modelling [4,5], property optimization [6,7] and knowledge discovery [8,9]. Its success is thus largely contingent on the amount of data available and a lack of sufficient data can severely impair model accuracy.…”