It is proposed that a complete understanding of two-dimensional gravity and its emergence in random matrix models requires fully embracing both Wigner (statistics) and 't Hooft (geometry). Using non-perturbative definitions of random matrix models that yield various JT gravity and JT supergravity models on surfaces of arbitrary topology, Fredholm determinants are used to extract precise information about the spectra of discrete microstates that underlie the physics. Smooth spacetime geometry emerges only when the gaps between microstates are negligible, and there are two complementary mechanisms that achieve this in matrix models. A core idea suggested by the computations is that in each case, the matrix models point to a single distinguished discrete spectrum that characterizes the holographic dual of the (super) gravity theory, living on the disc. Only asymptotically does it reduce to the effectively continuous (super) Schwarzian result. Several physical consequences of this are elucidated, including a reassessment of the factorization puzzle. It is entirely resolved if matrix models are recognized not as models of a single JT gravity theory but as describing an ensemble of JT deformations, each of whose holographic dual is a matrix. This would immediately explain why there is also no failure of factorization for traditional higher dimensional holographic duals. Several lessons that may pertain to quantum gravity on non-trivial topology in higher dimensions are noted, and a thorough introduction to the non-perturbative matrix model techniques used is included.