Path sampling of recurrent neural networks by incorporating known physics

Abstract

Recurrent neural networks have seen widespread use in modeling dynamical systems in varied domains such as weather prediction, text prediction and several others. Often one wishes to supplement the experimentally observed dynamics with prior knowledge or intuition about the system. While the recurrent nature of these networks allows them to model arbitrarily long memories in the time series used in training, it makes it harder to impose prior knowledge or intuition through generic constraints. In this work, we present a path sampling approach based on principle of Maximum Caliber that allows us to include generic thermodynamic or kinetic constraints into recurrent neural networks. We show the method here for a widely used type of recurrent neural network known as long short-term memory network in the context of supplementing time series collected from different application domains. These include classical Molecular Dynamics of a protein and Monte Carlo simulations of an open quantum system continuously losing photons to the environment and displaying Rabi oscillations. Our method can be easily generalized to other generative artificial intelligence models and to generic time series in different areas of physical and social sciences, where one wishes to supplement limited data with intuition or theory based corrections.

Exploring and Learning the Universe of Protein Allostery Using Artificial Intelligence Augmented Biophysical and Computational Approaches

“…Using the principle of variational inference implemented through deep neural networks and a predictive information bottleneck concept, a recently introduced framework leverages short MD simulations to estimate the reaction coordinates and perform iterative biased simulations that can subsequently enhance exploration of conformational landscapes and reliable inference of the associated thermodynamic and kinetic characteristics of the system. , Moreover, AI-based State Predictive Information Bottleneck (SPIB) approach can reliably learn a reaction coordinate via a deep neural network even from short and under-sampled trajectories . Further developments of these concepts produced a path sampling approach that integrates generic thermodynamic or kinetic constraints into long short-term memory (LSTM) networks to accurately learn time series such as MD trajectories for systems from different application domains . Going forward, the developments of these integrative biophysical approaches that leverage AI and ML tools to represent physics-based thermodynamic and kinetic drivers of efficient sampling in the form of neural networks would have significant implications for “autonomous” mapping of conformational landscapes, monitoring of allosteric changes, and detection of functional allosteric states.…”

“…93 Further developments of these concepts produced a path sampling approach that integrates generic thermodynamic or kinetic constraints into long shortterm memory (LSTM) networks to accurately learn time series such as MD trajectories for systems from different application domains. 94 Going forward, the developments of these integrative biophysical approaches that leverage AI and ML tools to represent physics-based thermodynamic and kinetic drivers of efficient sampling in the form of neural networks would have significant implications for "autonomous" mapping of conformational landscapes, monitoring of allosteric changes, and detection of functional allosteric states. MSM approaches are powerful tools for exploring long-time dynamic changes underlying the function of many allosterically regulated proteins, allowing for detailed network maps of functional states on the conformational landscape and quantitative analysis of the effect of perturbations on the thermodynamics and kinetics of allosteric transitions.…”

Memory Unlocks the Future of Biomolecular Dynamics: Transformative Tools to Uncover Physical Insights Accurately and Efficiently

“…More recently, it was shown that subsampling the LSTM-predicted paths subject to physical constraints based on the maximum caliber principle and retraining the LSTM model on this subset of paths could lead to improved quality in obtaining thermodynamic and kinetic properties of biomolecular systems. 116 As the recurrent neural network approach was originally developed for one-dimensional natural language processing, we expect that this LSTM approach alone may perform optimally on one-dimensional data. 117 Nevertheless, the LSTM architecture can be incorporated into a larger framework to perform complex multidimensional tasks.…”

Memory unlocks the future of biomolecular dynamics: Transformative tools to uncover physical insights accurately and efficiently

“…More recently, it was shown that subsampling the LSTMpredicted paths subject to physical constraints based on the Maximum Caliber principle and re-training the LSTM model on this subset of paths could lead to improved quality in obtaining thermodynamic and kinetic properties of biomolecular systems. 116 As the recurrent neural network approach was originally developed for one-dimensional natural language processing, we expect that this LSTM approach alone may perform optimally on one-dimensional data. 117 Nevertheless, the LSTM architecture can be incorporated into a larger framework to perform complex multidimensional tasks.…”