“…For more detailed overviews and further references on SGD optimization schemes we refer, e.g., to [8], [18,Section 1.1], [23, Section 1], and [39]. The effect of random initializations in the training of ANNs was studied, e.g., in [6,20,21,25,32,42] and the references mentioned therein. Another promising branch of research has investigated the convergence of SGD for the training of ANNs in the so-called overparametrized regime, where the number of ANN parameters has to be sufficiently large.…”