We consider a system of N servers inter-connected by some underlying graph topology G N . Tasks with unit-mean exponential processing times arrive at the various servers as independent Poisson processes of rate λ. Each incoming task is irrevocably assigned to whichever server has the smallest number of tasks among the one where it appears and its neighbors in G N .The above model arises in the context of load balancing in large-scale cloud networks and data centers, and has been extensively investigated in the case G N is a clique. Since the servers are exchangeable in that case, mean-field limits apply, and in particular it has been proved that for any λ < 1, the fraction of servers with two or more tasks vanishes in the limit as N → ∞. For an arbitrary graph G N , mean-field techniques break down, complicating the analysis, and the queue length process tends to be worse than for a clique. Accordingly, a graph G N is said to be N-optimal or √ N-optimal when the queue length process on G N is equivalent to that on a clique on an N-scale or √ N-scale, respectively. We prove that if G N is an Erdős-Rényi random graph with average degree d(N), then with high probability it is N-optimal and √ N-optimal if d(N) → ∞ and d(N)/( √ N log(N)) → ∞ as N → ∞, respectively. This demonstrates that optimality can be maintained at N-scale and √ N-scale while reducing the number of connections by nearly a factor N and √ N/ log(N) compared to a clique, provided the topology is suitably random. It is further shown that if G N contains Θ(N) bounded-degree nodes, then it cannot be N-optimal. In addition, we establish that an arbitrary graph G N is N-optimal when its minimum degree is N − o(N), and may not be N-optimal even when its minimum degree is cN + o(N) for any 0 < c < 1/2. Simulation experiments are conducted for various scenarios to corroborate the asymptotic results. * d.mukherjee@tue.nl; † s.c.borst@tue.nl; ‡ j.s.h.v.leeuwaarden@tue.nl arXiv:1707.05866v2 [math.PR] 6 Apr 2019Related work. The above model has been studied in [11,28], focusing on certain fixed-degree graphs and in particular ring topologies. The results demonstrate that the flexibility to forward tasks to a few neighbors, or even just one, with possibly shorter queues significantly improves the performance in terms of the waiting time and tail distribution of the queue length. This resembles the so-called 'power-of-two' effect in the classical case of a complete graph where tasks are assigned to the shortest queue among d servers selected uniformly at random. As shown by Mitzenmacher [16,17] and Vvedenskaya et al. [31], such a 'power-of-d' scheme provides a huge performance improvement over purely random assignment, even when d = 2, in particular super-exponential tail decay, translating into far better waiting-time performance. Further related