To understand typical dynamics of an open quantum system in continuous time, we introduce an ensemble of random Lindblad operators, which generate Markovian completely positive evolution in the space of density matrices. Spectral properties of these operators, including the shape of the spectrum in the complex plane, are evaluated by using methods of free probabilities and explained with non-Hermitian random matrix models. We also demonstrate universality of the spectral features. The notion of ensemble of random generators of Markovian qauntum evolution constitutes a step towards categorization of dissipative quantum chaos.

We study the time evolution of Ginibre matrices whose elements undergo Brownian motion. The non-Hermitian character of the Ginibre ensemble binds the dynamics of eigenvalues to the evolution of eigenvectors in a non-trivial way, leading to a system of coupled nonlinear equations resembling those for turbulent systems. We formulate a mathematical framework allowing simultaneous description of the flow of eigenvalues and eigenvectors, and we unravel a hidden dynamics as a function of new complex variable, which in the standard description is treated as a regulator only. We solve the evolution equations for large matrices and demonstrate that the non-analytic behavior of the Green's functions is associated with a shock wave stemming from a Burgers-like equation describing correlations of eigenvectors. We conjecture that the hidden dynamics, that we observe for the Ginibre ensemble, is a general feature of non-Hermitian random matrix models and is relevant to related physical applications. On the other hand, the original Dyson's idea of Brownian walk of real eigenvalues [11] interacting with a twodimensional Coulombic force still leads to novel insights. Examples include the study of determinantal processes [12][13][14], Loewner diffusion [15] or the fluctuations of nonintersecting interfaces in thermal equilibrium [16]. The concept of stochastic evolution of matrices has been recently exploited by several authors [17][18][19][20]. In particular, it was shown that the derivatives of the logarithms of characteristic determinants of diffusing GUE (Gaussian Unitary Ensemble), LUE (Laguerre Unitary Ensemble or Wishart Ensemble) and CUE (Circular Unitary Ensemble) obey Burgers-like nonlinear equations, where the role of viscosity is played by the inverse of the matrix size. For infinite dimensions of the matrix, these equations correspond to the inviscid regime and describe an evolution of the associated resolvents. Due to nonlinearity, they develop singularities (shock waves), whose positions correspond to the endpoints of the spectra. For matrices of finite size, the expansion around the shock wave solution of the initial viscid Burgers equation leads to a universal scaling of characteristic polynomials (and of the inverse characteristic polynomials as well), resulting in well known universal oscillatory behavior of the Airy, Bessel or Pearcey type. This approach has prompted, in particular, new perception of weak/strong coupling transition in multicolor Yang-Mills theory [21,22] and of the spontaneous breakdown of chiral symmetry in Euclidean QCD [23].In this letter, we unveil the intertwined evolution of eigenvalues and eigenvectors of stochastically evolving non-Hermitian matrices. To this end, we apply Dyson's idea to study diffusing Gaussian matrices for the case of the Ginibre Ensemble (GE). The central object of the paper is a generalized averaged characteristic polynomial. Its logarithmic derivatives, which contain the information about both the eigenvalues and eigenvectors of the evolving matrix, fulfill a sys...

We extend the so-called "single ring theorem" [1], also known as the Haagerup-Larsen theorem [2], by showing that in the limit when the size of the matrix goes to infinity a particular correlator between left and right eigenvectors of the relevant non-hermitian matrix X, being the spectral density weighted by the squared eigenvalue condition number, is given by a simple formula involving only the radial spectral cumulative distribution function of X. We show that this object allows to calculate the conditional expectation of the squared eigenvalue condition number. We give examples and we provide cross-check of the analytic prediction by the large scale numerics.

Following our recent letter [1], we study in detail an entry-wise diffusion of non-hermitian complex matrices. We obtain an exact partial differential equation (valid for any matrix size N and arbitrary initial conditions) for evolution of the averaged extended characteristic polynomial. The logarithm of this polynomial has an interpretation of a potential which generates a Burgers dynamics in quaternionic space. The dynamics of the ensemble in the large N limit is completely determined by the coevolution of the spectral density and a certain eigenvector correlation function. This coevolution is best visible in an electrostatic potential of a quaternionic argument built of two complex variables, the first of which governs standard spectral properties while the second unravels the hidden dynamics of eigenvector correlation function. We obtain general formulas for the spectral density and the eigenvector correlation function for large N and for any initial conditions. We exemplify our studies by solving three examples, and we verify the analytic form of our solutions with numerical simulations.

We develop an exact formalism to study how network architecture influences the transient dynamics of large dynamical systems, described by a set of randomly coupled linear differential equations, in the vicinity of a stationary point. We show that for unidirectional networks the average dynamical response to initial perturbations is universal and only depends on a single parameter, encoding the average interaction strength between the individual constituents. We illustrate our results with numerical simulations of large systems with different types of disorder.

Using large N arguments, we propose a scheme for calculating the two-point eigenvector correlation function for non-normal random matrices in the large N limit. The setting generalizes the quaternionic extension of free probability to two-point functions. In the particular case of biunitarily invariant random matrices, we obtain a simple, general expression for the two-point eigenvector correlation function, which can be viewed as a further generalization of the single ring theorem. This construction has some striking similarities to the freeness of the second kind known for the Hermitian ensembles in large N . On the basis of several solved examples, we conjecture two kinds of microscopic universality of the eigenvectors -one in the bulk, and one at the rim. The form of the conjectured bulk universality agrees with the scaling limit found by Chalker and Mehlig [JT Chalker, B Mehlig, Phys. Rev. Lett. 81 (1998) 3367] in the case of the complex Ginibre ensemble.

No abstract

We demonstrate that in residual neural networks (ResNets) dynamical isometry is achievable irrespective of the activation function used. We do that by deriving, with the help of Free Probability and Random Matrix Theories, a universal formula for the spectral density of the input-output Jacobian at initialization, in the large network width and depth limit. The resulting singular value spectrum depends on a single parameter, which we calculate for a variety of popular activation functions, by analyzing the signal propagation in the artificial neural network. We corroborate our results with numerical simulations of both random matrices and ResNets applied to the CIFAR-10 classification problem. Moreover, we study consequences of this universal behavior for the initial and late phases of the learning processes. We conclude by drawing attention to the simple fact, that initialization acts as a confounding factor between the choice of activation function and the rate of learning. We propose that in ResNets this can be resolved based on our results by ensuring the same level of dynamical isometry at initialization. I. INTRODUCTIONDeep Learning has achieved unparalleled success in fields such as object detection and recognition, language translation, and speech recognition [16]. At the same time, models achieving these state-of-the-art results are increasingly deep and complex [3], which often leads to optimization challenges such as vanishing gradients. Many solutions to this problem have been proposed. In particular, Residual Neural Networks remedy this to some extent [10,35] by using skip connections in the network architecture, which improve gradient flow. As a result, Residual Neural Networks outmatched other competing models in the 2015 ILSVRC and COCO competitions. Yet another approach towards solving this problem is to tailor fit the networks weight initialization to facilitate training, for example by ensuring dynamical isometry [28]. In this latter case, the insights are based on an analysis of the statistical properties of information propagation in the network and a study of the full singular spectrum of a particular matrix, namely the input-output Jacobian, via the techniques of Free Probability and Random Matrix Theories (FPT & RMT). This perspective has recently led to successfully training a 10000 layer vanilla convolutional neural network [38].RMT is a versatile tool that, since its inception, saw a substantial share of applications, from the earliest in nuclear physics [37] to the latest in game theory [4] (see [1] for some of the use cases discovered in the mean time). It is thus not surprising that it found its way to be used to understand artificial neural networks. In particular, to study their loss surface [6,27], the associated Gram matrix [19,30] and in the case of single layer networks, their dynamics [17]. Our main contribution is extending the theoretical analysis of [28,29,33] to residual networks. In particular, we find that residual networks can achieve dynamical isometry for many differe...