Gradient descent dynamics and the jamming transition in infinite dimensions

Abstract

Gradient descent dynamics in complex energy landscapes featuring multiple minima finds application in many different problems, from soft matter to machine learning. Here, we analyze one of the simplest examples, namely that of soft repulsive particles in the limit of infinite spatial dimension d. The gradient descent dynamics then displays a jamming transition: at low density, it reaches zero-energy states in which particles' overlaps are fully eliminated, while at high density the energy remains finite and overlaps persist. At the transition the dynamics become critical. In the d → ∞ limit, the dynamics can be written in terms of a set of self-consistent equations. We analyze these equations and we present some partial progress towards their solution. We also study the Random Lorentz Gas in a range of d = 2 . . . 22, and obtain a robust estimate for the jamming transition in d → ∞. The jamming transition is analogous to the capacity transition in supervised learning, and in the appendix we discuss this analogy in the case of a simple one-layer fully-connected perceptron.

Dependence of the glass transition and jamming densities on spatial dimension

“…3(d), we compare the scaled jamming densities we obtain with those in [51]. We also show the recent theoretical calculations in [15] for the corresponding quantity φ th , which shows the same trend as our φ J data, but are smaller (reasons why this may be expected are discussed in [52]). We further show the φ 0 values we obtain, along with the corresponding calculated values (Kauzmann density φ K ) in [15].…”