Optimal control of molecular dynamics using Markov state models

Abstract

A numerical scheme for solving high-dimensional stochastic control problems on an infinite time horizon that appear relevant in the context of molecular dynamics is outlined. The scheme rests on the interpretation of the corresponding Hamilton-Jacobi-Bellman equation as a nonlinear eigenvalue problem that, using a logarithmic transformation, can be recast as a linear eigenvalue problem, for which the principal eigenvalue and its eigenfunction are sought. The latter can be computed e ciently by approximating the underlying stochastic process with a coarse-grained Markov state model for the dominant metastable sets. We illustrate our method with two numerical examples, one of which involves the task of maximizing the population of ↵-helices in an ensemble of small biomolecules (Alanine dipeptide), and discuss the relation to the large deviation principle of Donsker and Varadhan.

Tackling Exascale Software Challenges in Molecular Dynamics Simulations with GROMACS

“…1. To adapt to this reality, researchers are increasingly using large ensembles of simulations, either to simply sample better, or new algorithms such as replica exchange simulation, 24 Markov state models, 22 or milestoning 13 that analyze and exchange data between multiple simulations to improve overall sampling. In many cases, this achieves as much as two-fold superscaling, i.e., an ensemble of 100 simulations running on 10 nodes each might provide the same sampling efficiency as a single simulation running on 2000 cores.…”

Optimal control of multiscale systems using reduced-order models

“…For the control problem (49)- (50), the analog of (8) for the case of an infinitetime horizon with time-averaged cost and unbounded domain reads [54,55] …”

“…withĀ andB as in (55). Without loss of generality, we have ignored the additive constant Q in the cost term that appears in the homogenized eigenvalue equation (54).…”

“…Stochastic control problems appear in a variety of applications, such as statistics [17,16], financial mathematics [15,53], molecular dynamics [55,28] and materials science [57,6], to mention just a few. A common feature of the models used is that they are high-dimensional and possess several characteristic time scales.…”

Variational Characterization of Free Energy: Theory and Algorithms

“…The strength of the methods described below is that they can be combined with model reduction methods such as averaging, homogenization or Markov state modeling if either suitable collective variables, a reaction coordinate or some dominant metastable sets are known; see, e.g., [7,15,29,32,36,37] for the general approach and the application to molecular dynamics.…”