Efficient and Robust Emergence of Conventions through Learning and Staying

Understanding and improving zeroth-order optimization methods on AI-driven molecule optimization

“…Specifically, we evaluate several ZO optimization methods for the problem of molecule optimization in terms of convergence speed, convergence accuracy, and robustness to the unusual function landscapes (described further in Section 2.4) of molecular objectives. Our experiments on molecule optimization tasks from Guacamol show that ZO-GD underperforms other ZO methods, while ZO-signGD 11 performs comparably and in several cases better than ZO-Adam despite being known to have worse convergence accuracy than ZO-Adam for other problems like adversarial attacks. 11 Our results indicate that the sign operation may potentially increase robustness to the function landscapes of molecular objectives.…”

“…Our experiments on molecule optimization tasks from Guacamol show that ZO-GD underperforms other ZO methods, while ZO-signGD 11 performs comparably and in several cases better than ZO-Adam despite being known to have worse convergence accuracy than ZO-Adam for other problems like adversarial attacks. 11 Our results indicate that the sign operation may potentially increase robustness to the function landscapes of molecular objectives. Furthermore, we provide insights into practical application of ZO optimization in drug discovery scenarios for both lead optimization tasks and the discovery of novel molecules, as well as propose the use of a hybrid approach combining others models with QMO.…”

“…Sign-based gradient descent is known to be effective in achieving fast convergence speed in stochastic settings: in the stochastic rst-order oracle setting, Bernstein et al 25 showed that signSGD could have faster empirical convergence speed than SGD, and in the zeroth-order stochastic setting, Liu et al 11 similarly showed that ZO-signSGD has faster convergence speed than many ZO optimization methods at the cost of worse accuracy (i.e., converging only to the neighborhood of an optima). The fast convergence of sign-based methods is motivated by the idea that the sign operation is more robust to stochastic noise, and though our formulation of molecule optimization is non-stochastic, the sign operation is potentially more robust to the difficult landscapes of molecular objective functions.…”

“…2. ZO sign-based gradient descent (ZO-signGD) : 11 analogous to sign-based SGD (signSGD) 25 in the first-order stochastic setting. ZO-signGD uses the same point updating rule as ZO-GD but instead uses the sign of the current estimate as the descent direction , where sign(·) denotes the element-wise sign operation.…”

“…ZO optimization is a class of methods used for solving black-box problems by estimating gradients using only zeroth-order function evaluations and performing iterative updates as in first-order methods like gradient descent (GD). 9 Many types of ZO optimization algorithms have been developed, including the ZO gradient descent (ZO-GD), 10 ZO sign-based gradient descent (ZO-signGD), 11 ZO adaptive momentum method (ZO-AdaMM, or ZO-Adam specifically for the Adam variant), 12 and more. 13,14 The optimality of ZO optimization methods has also been studied under given problem assumptions.…”