We consider sequences of graphs (G n ) and define various notions of convergence related to these sequences: "left convergence" defined in terms of the densities of homomorphisms from small graphs into G n ; "right convergence" defined in terms of the densities of homomorphisms from G n into small graphs; and convergence in a suitably defined metric.In Part I of this series, we show that left convergence is equivalent to convergence in metric, both for simple graphs G n , and for graphs G n with nodeweights and edgeweights. One of the main steps here is the introduction of a cut-distance comparing graphs, not necessarily of the same size. We also show how these notions of convergence provide natural formulations of Szemerédi partitions, sampling and testing of large graphs.

Diffusion is a fundamental graph process, underpinning such phenomena as epidemic disease contagion and the spread of innovation by word-of-mouth. We address the algorithmic problem of finding a set of k initial seed nodes in a network so that the expected size of the resulting cascade is maximized, under the standard independent cascade model of network diffusion. Runtime is a primary consideration for this problem due to the massive size of the relevant input networks.We provide a fast algorithm for the influence maximization problem, obtaining the nearoptimal approximation factor of (1 − 1 e − ǫ), for any ǫ > 0, in time O((m + n)kǫ −2 log n). Our algorithm is runtime-optimal (up to a logarithmic factor) with respect to network size, and substantially improves upon the previously best-known algorithms which run in time Ω(mnk · POLY(ǫ −1 )). Furthermore, our algorithm can be modified to allow early termination: if it is terminated after O(β(m + n)k log n) steps for some β < 1 (which can depend on n), then it returns a solution with approximation factor O(β). Finally, we show that this runtime is optimal (up to logarithmic factors) for any β and fixed seed size k.

No abstract

This paper proposes a new optimization algorithm called Entropy-SGD for training deep neural networks that is motivated by the local geometry of the energy landscape. Local extrema with low generalization error have a large proportion of almost-zero eigenvalues in the Hessian with very few positive or negative eigenvalues. We leverage upon this observation to construct a local-entropy-based objective function that favors well-generalizable solutions lying in large flat regions of the energy landscape, while avoiding poorly-generalizable solutions located in the sharp valleys. Conceptually, our algorithm resembles two nested loops of SGD where we use Langevin dynamics in the inner loop to compute the gradient of the local entropy before each update of the weights. We show that the new objective has a smoother energy landscape and show improved generalization over SGD using uniform stability, under certain assumptions. Our experiments on convolutional and recurrent networks demonstrate that Entropy-SGD compares favorably to state-of-the-art techniques in terms of generalization error and training time.

No abstract

We consider sequences of graphs (Gn) and define various notions of convergence related to these sequences including "left-convergence," defined in terms of the densities of homomorphisms from small graphs into Gn, and "right-convergence," defined in terms of the densities of homomorphisms from Gn into small graphs.We show that right-convergence is equivalent to left-convergence, both for simple graphs Gn, and for graphs Gn with nontrivial nodeweights and edgeweights. Other equivalent conditions for convergence are given in terms of fundamental notions from combinatorics, such as maximum cuts and Szemerédi partitions, and fundamental notions from statistical physics, like energies and free energies. We thereby relate local and global properties of graph sequences. Quantitative forms of these results express the relationships among different measures of similarity of large graphs.

We study random subgraphs of an arbitrary finite connected transitive graph G obtained by independently deleting edges with probability 1 − p. Let V be the number of vertices in G, and let Ω be their degree. We define the critical threshold p c = p c (G, λ) to be the value of p for which the expected cluster size of a fixed vertex attains the value λV 1/3 , where λ is fixed and positive. We show that for any such model, there is a phase transition at p c analogous to the phase transition for the random graph, provided that a quantity called the triangle diagram is sufficiently small at the threshold p c . In particular, we show that the largest cluster inside a scaling window of size |p − p c | = Θ(Ω −1 V −1/3 ) is of size Θ(V 2/3 ), while below this scaling window, it is much smaller, of order O(ǫ −2 log(V ǫ 3 )), with ǫ = Ω(p c − p). We also obtain an upper bound O(Ω(p − p c )V ) for the expected size of the largest cluster above the window. In addition, we define and analyze the percolation probability above the window and show that it is of order Θ(Ω(p − p c )). Among the models for which the triangle diagram is small enough to allow us to draw these conclusions are the random graph, the n-cube and certain Hamming cubes, as well as the spread-out n-dimensional torus for n > 6.

In a previous paper, we defined a version of the percolation triangle condition that is suitable for the analysis of bond percolation on a finite connected transitive graph, and showed that this triangle condition implies that the percolation phase transition has many features in common with the phase transition on the complete graph. In this paper, we use a new and simplified approach to the lace expansion to prove quite generally that for finite graphs that are tori the triangle condition for percolation is implied by a certain triangle condition for simple random walks on the graph.The latter is readily verified for several graphs with vertex set {0, 1, . . . , r − 1} n , including the Hamming cube on an alphabet of r letters (the n-cube, for r = 2), the n-dimensional torus with nearest-neighbor bonds and n sufficiently large, and the n-dimensional torus with n > 6 and sufficiently spread-out (long range) bonds. The conclusions of our previous paper thus apply to the percolation phase transition for each of the above examples.