In this paper, we present a simple combinatorial algorithm that solves symmetric diagonally dominant (SDD) linear systems in nearly-linear time. It uses very little of the machinery that previously appeared to be necessary for a such an algorithm. It does not require recursive preconditioning, spectral sparsification, or even the Chebyshev Method or Conjugate Gradient. After constructing a "nice" spanning tree of a graph associated with the linear system, the entire algorithm consists of the repeated application of a simple (non-recursive) update rule, which it implements using a lightweight data structure. The algorithm is numerically stable and can be implemented without the increased bit-precision required by previous solvers. As such, the algorithm has the fastest known running time under the standard unit-cost RAM model. We hope that the simplicity of the algorithm and the insights yielded by its analysis will be useful in both theory and practice.

In this paper, we introduce a new framework for approximately solving flow problems in capacitated, undirected graphs and apply it to provide asymptotically faster algorithms for the maximum s-t flow and maximum concurrent multicommodity flow problems. For graphs with n vertices and m edges, it allows us to find an ε-approximate maximum s-t flow in time O(m 1+o(1) ε −2 ), improving on the previous best bound of O(mn 1/3 poly(ε −1 )). Applying the same framework in the multicommodity setting solves a maximum concurrent multicommodity flow problem withOur algorithms utilize several new technical tools that we believe may be of independent interest:• We give a non-Euclidean generalization of gradient descent and provide bounds on its performance. Using this, we show how to reduce approximate maximum flow and maximum concurrent flow to oblivious routing.• We define and provide an efficient construction of a new type of flow sparsifier. Previous sparsifier constructions approximately preserved the size of cuts and, by duality, the value of the maximum flows as well. However, they did not provide any direct way to route flows in the sparsifier G back in the original graph G, leading to a longstanding gap between the efficacy of sparsification on flow and cut problems. We ameliorate this by constructing a sparsifier G that can be embedded (very efficiently) into G with low congestion, allowing one to transfer flows from G back to G.• We give the first almost-linear-time construction of an O(m o(1) )-competitive oblivious routing scheme. No previous such algorithm ran in time better than Ω(mn). By reducing the running time to almost-linear, our work provides a powerful new primitive for constructing very fast graph algorithms.The interested reader is referred to the full version of the paper [8] for a more complete treatment of these results.

We study the design of nearly-linear-time algorithms for approximately solving positive linear programs. Both the parallel and the sequential deterministic versions of these algorithms require O(ε −4 ) iterations, a dependence that has not been improved since the introduction of these methods in 1993 by Luby and Nisan. Moreover, previous algorithms and their analyses rely on update steps and convergence arguments that are combinatorial in nature, and do not seem to arise naturally from an optimization viewpoint. In this paper, we leverage insights from optimization theory to construct a novel algorithm that breaks the longstanding O(ε −4 ) barrier. Our algorithm has a simple analysis and a clear motivation. Our work introduces a number of novel techniques, such as the combined application of gradient descent and mirror descent, and a truncated, smoothed version of the standard multiplicative weight update, which may be of independent interest.

Positive linear programs (LP), also known as packing and covering linear programs, are an important class of problems that bridges computer science, operation research, and optimization. Efficient algorithms for solving such LPs have received significant attention in the past 20 years [2,3,4,6,7,9,11,15,16,18,19,21,24,25,26,29,30]. Unfortunately, all known nearly-linear time algorithms for producing (1+ε)-approximate solutions to positive LPs have a running time dependence that is at least proportional to ε −2 . This is also known as an O(1/ √ T ) convergence rate and is particularly poor in many applications.In this paper, we leverage insights from optimization theory to break this longstanding barrier. Our algorithms solve the packing LP in time O(N ε −1 ) and the covering LP in time O(N ε −1.5 ). At high level, they can be described as linear couplings of several first-order descent steps. This is the first application of our linear coupling technique (see [1]) to problems that are not amenable to blackbox applications known iterative algorithms in convex optimization. Our work also introduces a sequence of new techniques, including the stochastic and the non-symmetric execution of gradient truncation operations, which may be of independent interest.

In this paper, we provide a novel construction of the linear-sized spectral sparsifiers of Batson, Spielman and Srivastava [BSS14]. While previous constructions required Ω(n 4 ) running time [BSS14, Zou12], our sparsification routine can be implemented in almost-quadratic running time O(n 2+ε ). The fundamental conceptual novelty of our work is the leveraging of a strong connection between sparsification and a regret minimization problem over density matrices. This connection was known to provide an interpretation of the randomized sparsifiers of Spielman and Srivastava [SS11] via the application of matrix multiplicative weight updates (MWU) [CHS11,Vis14]. In this paper, we explain how matrix MWU naturally arises as an instance of the Follow-theRegularized-Leader framework and generalize this approach to yield a larger class of updates. This new class allows us to accelerate the construction of linear-sized spectral sparsifiers, and give novel insights on the motivation behind Batson, Spielman and Srivastava [BSS14].

First-order methods play a central role in large-scale machine learning. Even though many variations exist, each suited to a particular problem, almost all such methods fundamentally rely on two types of algorithmic steps: gradient descent, which yields primal progress, and mirror descent, which yields dual progress.We observe that the performances of gradient and mirror descent are complementary, so that faster algorithms can be designed by linearly coupling the two. We show how to reconstruct Nesterov's accelerated gradient methods using linear coupling, which gives a cleaner interpretation than Nesterov's original proofs. We also discuss the power of linear coupling by extending it to many other settings that Nesterov's methods cannot apply to. * The authors would like to thank Silvio Micali for listening to our work and suggesting the name "linear coupling". 1 Here, variable x is constrained to lie in a convex set Q ⊆ R n , which is known as the constraint set of the problem. 2 We emphasize here that these two terms are sometimes used ambiguosly in the literature; in this paper, we attempt to stick as close as possible to the conventions of the optimization community and in particular in the textbooks [9,26] with one exception: we extend the definition of gradient descent to non-Euclidean norms in a natural way, following [18].

We give a novel spectral approximation algorithm for the balanced separator problem that, given a graph G, a constant balance b ∈ (0, 1/2], and a parameter γ, either finds an Ω(b)balanced cut of conductance O( √ γ) in G, or outputs a certificate that all b-balanced cuts in G have conductance at least γ, and runs in timeÕ(m). This settles the question of designing asymptotically optimal spectral algorithms for balanced separator. Our algorithm relies on a variant of the heat kernel random walk and requires, as a subroutine, an algorithm to compute exp(−L)v where L is the Laplacian of a graph related to G and v is a vector. Algorithms for computing the matrix-exponential-vector product efficiently comprise our next set of results. Our main result here is a new algorithm which computes a good approximation to exp(−A)v for a class of symmetric positive semidefinite (PSD) matrices A and a given vector u, in time roughlyÕ(m A ), where m A is the number of non-zero entries of A. This uses, in a non-trivial way, the breakthrough result of Spielman and Teng on inverting symmetric and diagonally-dominant matrices inÕ(m A ) time. Finally, we prove that e −x can be uniformly approximated up to a small additive error, in a non-negative interval [a, b] with a polynomial of degree roughly √ b − a. While this result is of independent interest in approximation theory, we show that, via the Lanczos method from numerical analysis, it yields a simple algorithm to compute exp(−A)v for symmetric PSD matrices that runs in time roughly O(t A · A ), where t A is time required for the computation of the vector Aw for given vector w. As an application, we obtain a simple and practical algorithm, with output conductance O( √ γ), for balanced separator that runs in timeÕ( m / √ γ). This latter algorithm matches the running time, but improves on the approximation guarantee of the Evolving-Sets-based algorithm by Andersen and Peres for balanced separator.

No abstract