Langevin Dynamics with Variable Coefficients and Nonconservative Forces: From Stationary States to Numerical Methods

Sachs, Matthias; Leimkuhler, Benedict; Danos, Vincent

doi:10.3390/e19120647

Cited by 29 publications

(33 citation statements)

References 43 publications

(75 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The methods we propose thus contain additive noise (which has an adjustable but fixed strength) and, in practice a second term with an unknown covariance arising from the gradient approximation. Such an approach is close to the systems treated in [33] where the SDEs take the form of a Langevin system dq = pdt, dp = F (q)dt − Γ(q)pdt + Σ(q)dW.…”

Section: Sde-based Schemes In Machine Learningmentioning

confidence: 95%

“…Nondegeneracy of Σ(q) is required for the results of [33] to hold, but we will obtain that by driving the system by additive noise of defined strength (in each momentum equation). In the case of the method AdLaLa, described below, we only have the hypocoercivity results [39] which can be used directly to justify the method.…”

Section: Sde-based Schemes In Machine Learningmentioning

confidence: 99%

See 1 more Smart Citation

Partitioned integrators for thermodynamic parameterization of neural networks

Leimkuhler¹,

Matthews²,

Vlaar³

2019

Foundations of Data Science

Self Cite

View full text Add to dashboard Cite

Traditionally, neural networks are parameterized using optimization procedures such as stochastic gradient descent, RMSProp and ADAM. These procedures tend to drive the parameters of the network toward a local minimum. Various algorithmic devices have been introduced to enhance (or, in some cases, delay) the convergence of the optimizer, to increase the exploration of low loss states. These methods ultimately provide a crude sampling for a probability distribution on parameter space. As an alternative, one may derive "sampling" algorithms (referred to here as "thermodynamic parameterization methods") for a defined target distribution on parameter space. By using state-of-theart discretizations of stochastic differential equations with a known invariant probability distribution, we argue that it is much easier to control the properties of the distribution. Stochastic Gradient Langevin Dynamics, the "unadjusted Langevin algorithm", and Adaptive Langevin Dynamics (also known as Stochastic Gradient Nosé-Hoover dynamics) are examples of existing thermodynamic parameterization methods in use for machine learning, but these can be substantially improved. We find that by partitioning the parameters based on natural layer structure we obtain schemes with rapid convergence for data sets with complicated loss landscapes. We describe easy-to-implement hybrid partitioned numerical algorithms, based on discretized stochastic differential equations, which are adapted to feed-forward neural networks, including LaLa (a multi-layer Langevin algorithm), AdLaLa (combining the adaptive Langevin and Langevin algorithms) and LOL (combining Langevin and Overdamped Langevin); we examine the convergence of these methods using numerical studies and compare their performance among themselves and in relation to standard alternatives such as stochastic gradient descent and ADAM. We present evidence that thermodynamic parameterization methods can be (i) faster, (ii) more accurate, and (iii) more robust than standard algorithms incorporated into machine learning frameworks, in particular for data sets with complicated loss landscapes. Moreover, we show in numerical studies that methods based on sampling excite many degrees of freedom. The equipartition property, which is a consequence of their ergodicity, means that these methods keep in play an ensemble of low-loss states during the training process. By drawing parameter states from a sufficiently rich distribution of nearby candidate states, we show that the thermodynamic schemes produce smoother classifiers, improve generalization and reduce overfitting compared to traditional optimizers.

show abstract

Section: Sde-based Schemes In Machine Learningmentioning

confidence: 95%

Section: Sde-based Schemes In Machine Learningmentioning

confidence: 99%

Partitioned integrators for thermodynamic parameterization of neural networks

Leimkuhler¹,

Matthews²,

Vlaar³

2019

Foundations of Data Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…In this section, we show results for the current computed along a given trajectory of the bond-length timeevolution obtained from the solution of the Langevin equation. To compute a trajectory x(t), we utilize an m-BAOAB algorithm provided by Sachs et al 76 , which enables a numerical solution of the Langevin equation with a coordinate dependent viscosity and diffusion coefficient. The trajectory is used to compute Green's functions, and current with first order dynamical corrections using the equation presented in section III A.…”

Section: Currentmentioning

confidence: 99%

Current-induced atomic motion, structural instabilities, and negative temperatures on molecule-electrode interfaces in electronic junctions

2020

View full text Add to dashboard Cite

Molecule-electrode interfaces in molecular electronic junctions are prone to chemical reactions, structural changes, and localized heating effects caused by electric current. These can be exploited for device functionality or may be degrading processes that limit performance and device lifetime. We develop a nonequilibrium Green's function based transport theory in which the central region atoms and, more importantly, atoms on molecule-electrode interfaces are allowed to move. The separation of time-scales of slow nuclear motion and fast electronic dynamics enables the algebraic solution of the Kadanoff-Baym equations in the Wigner space. As a result, analytical expressions for dynamical corrections to the adiabatically computed Green's functions are produced. These dynamical corrections depend not only on the instantaneous molecular geometry but also on the nuclear velocities. To make the theoretical approach fully self-consistent, the same time-separation approach is used to develop expressions for the adiabatic, dissipative, and stochastic components of current-induced forces in terms of adiabatic Green's functions. Using these current induced forces, the equation of motion for the nuclear degrees of freedom is cast in the form of a Langevin equation. The theory is applied to model molecular electronic junctions. We observe that the interplay between the value of the spring constant for the molecule-electrode chemical bond and electronic coupling strength to the corresponding electrode is critical for the appearance of structural instabilities and, consequently, telegraphic switching in the electric current. The range of model parameters is identified to observe structurally stable molecular junctions as well as various different kinds of current-induced telegraphic switching. The interfacial structural instabilities are also quantified based on current noise calculations.

show abstract

“…The first is to add the solvent, which exerts a friction and random forces on each of the atoms of the macromolecule. In the implicit solvent model, also called Brownian dynamics, the effect of solvent is modeled by Langevin's equation (Oda et al, 2008;Sachs et al, 2017),…”

Section: Molecular-dynamics (Md) Simulationsmentioning

confidence: 99%

Survey of the analysis of continuous conformational variability of biological macromolecules by electron microscopy

Sorzano

Jiménez

Mota

et al. 2019

Acta Crystallogr F Struct Biol Commun

View full text Add to dashboard Cite

Single-particle analysis by electron microscopy is a well established technique for analyzing the three-dimensional structures of biological macromolecules. Besides its ability to produce high-resolution structures, it also provides insights into the dynamic behavior of the structures by elucidating their conformational variability. Here, the different image-processing methods currently available to study continuous conformational changes are reviewed.

show abstract

Langevin Dynamics with Variable Coefficients and Nonconservative Forces: From Stationary States to Numerical Methods

Cited by 29 publications

References 43 publications

Partitioned integrators for thermodynamic parameterization of neural networks

Partitioned integrators for thermodynamic parameterization of neural networks

Current-induced atomic motion, structural instabilities, and negative temperatures on molecule-electrode interfaces in electronic junctions

Survey of the analysis of continuous conformational variability of biological macromolecules by electron microscopy

Contact Info

Product

Resources

About