Reproducibility of Free Energy Calculations across Different Molecular Simulation Software Packages

Abstract

Alchemical free energy calculations are an increasingly important modern simulation technique to calculate free energy changes on binding or solvation. Contemporary molecular simulation software such as AMBER, CHARMM, GROMACS, and SOMD include support for the method. Implementation details vary among those codes, but users expect reliability and reproducibility, i.e., for a given molecular model and set of force field parameters, comparable free energy differences should be obtained within statistical bounds regardless of the code used. Relative alchemical free energy (RAFE) simulation is increasingly used to support molecule discovery projects, yet the reproducibility of the methodology has been less well tested than its absolute counterpart. Here we present RAFE calculations of hydration free energies for a set of small organic molecules and demonstrate that free energies can be reproduced to within about 0.2 kcal/mol with the aforementioned codes. Absolute alchemical free energy simulations have been carried out as a reference. Achieving this level of reproducibility requires considerable attention to detail and package-specific simulation protocols, and no universally applicable protocol emerges. The benchmarks and protocols reported here should be useful for the community to validate new and future versions of software for free energy calculations.

The SAMPL6 SAMPLing challenge: assessing the reliability and efficiency of binding free energy calculations

“…The latter, in particular, requires specifying force field parameters and partial charges, but also other components of the simulation, such as ion concentrations and the treatment of long-range interactions (e.g., PME, reaction field, Lennard-Jones cutoff, dispersion correction). Treating long-range interactions equivalently is particularly challenging due to differences in functional forms, implementations, and options supported by the various software packages, including small discrepancies in the value of the Coulomb constant [29,30]. Establishing a set of simulation settings that minimizes these differences does not prevent systematic bias due to sampling issues, but it makes it possible to detect by comparing calculations performed with independent methods and/or starting from different initial configurations.…”

Solvation Entropy Made Simple

“…They also suffer from inherent reproducibility issues due to the sensitivity of Newtonian dynamics to initial conditions and the lingering effects of such conditions if sampling is insufficient. 24,25 The purpose of this work is to formulate an efficient approximation for calculating molecular entropy in solution based only on physical and geometric arguments. Three models, S ω , S , and S α , are derived that rely only on such arguments-no empirically fitted parametersand elementary solvent properties (e.g., mass density).…”

The SAMPL6 SAMPLing challenge: Assessing the reliability and efficiency of binding free energy calculations

“…Similar observations were made in previous work in different contexts. In a reproducibility study involving four different implementations of relative hydration free energy calculations, the authors found in many cases statistically significant ΔΔ differences on the order of 0.2 kcal/mol [27]. Systematic differences of the same order of magnitude were detected in a recent study comparing Monte Carlo and Molecular Dynamics sampling for binding free energy calculations [24], although, in this case, differences in water models and periodic boundary conditions might confound the analysis.…”

“…PME, reaction field, Lennard-Jones cutoff, dispersion correction). Treating long-range interactions equivalently is particularly challenging because different software packages typically make implementation decisions with the goal of increased performance, and available options such as supported cutoff algorithms and options, the functional form of switching functions, or even the value of the Coulomb constant may differ for various reasons [26,27]. As a consequence, it may be necessary to compromise to establish simulation settings that minimize obvious differences.…”

“…The task is thus not trivial to carry out without collaborations between multiple groups. Previous work investigating the reproducibility of relative alchemical hydration free energy calculations across four molecular packages uncovered various issues and challenges in comparing across simulation packages and resulted in various bug fixes [27]. However, the reproducibility and efficiencies of various simulation-based approaches has not yet been evaluated in the context of binding free energy calculations, which is the focus of this work.…”