In this documentname, we introduce a notion called “approximate ultrametricity,” which encapsulates the phenomenology of a sequence of random probability measures having supports that behave like ultrametric spaces insofar as they decompose into nested balls. We provide a sufficient condition for a sequence of random probability measures on the unit ball of an infinite‐dimensional separable Hilbert space to admit such a decomposition, whose elements we call clusters. We also characterize the laws of the measures of the clusters by showing that they converge in law to the weights of a Ruelle probability cascade. These results apply to a large class of classical models in mean field spin glasses. We illustrate the notion of approximate ultrametricity by proving a conjecture of Talagrand regarding mixed p‐spin glasses that is known to imply a prediction of Dotsenko‐Franz‐Mézard. © 2017 Wiley Periodicals, Inc.