We give improved algorithms for the ℓ p -regression problem, min x x p such that Ax = b, for all p ∈ (1, 2) ∪ (2, ∞). Our algorithms obtain a high accuracy solution in O p (mwhere each iteration requires solving an m × m linear system, with m being the dimension of the ambient space.Incorporating a procedure for maintaining an approximate inverse of the linear systems that we need to solve at each iteration, we give algorithms for solving ℓ p -regression to 1/poly(n) accuracy that runs in time O p (m max{ω,7/3} ), where ω is the matrix multiplication constant. For the current best value of ω > 2.37, this means that we can solve ℓ p regression as fast as ℓ 2 regression, for all constant p bounded away from 1.Our algorithms can be combined with nearly-linear time solvers for linear systems in graph Laplacians to give minimum ℓ p -norm flow / voltage solutions to 1/poly(n) accuracy on an * This paper has been published at SODA 2019 [Adi+], and was initially

We show how to perform sparse approximate Gaussian elimination for Laplacian matrices. We present a simple, nearly linear time algorithm that approximates a Laplacian by a matrix with a sparse Cholesky factorization -the version of Gaussian elimination for symmetric matrices. This is the first nearly linear time solver for Laplacian systems that is based purely on random sampling, and does not use any graph theoretic constructions such as low-stretch trees, sparsifiers, or expanders. The crux of our analysis is a novel concentration bound for matrix martingales where the differences are sums of conditionally independent variables.

We give a novel spectral approximation algorithm for the balanced separator problem that, given a graph G, a constant balance b ∈ (0, 1/2], and a parameter γ, either finds an Ω(b)balanced cut of conductance O( √ γ) in G, or outputs a certificate that all b-balanced cuts in G have conductance at least γ, and runs in timeÕ(m). This settles the question of designing asymptotically optimal spectral algorithms for balanced separator. Our algorithm relies on a variant of the heat kernel random walk and requires, as a subroutine, an algorithm to compute exp(−L)v where L is the Laplacian of a graph related to G and v is a vector. Algorithms for computing the matrix-exponential-vector product efficiently comprise our next set of results. Our main result here is a new algorithm which computes a good approximation to exp(−A)v for a class of symmetric positive semidefinite (PSD) matrices A and a given vector u, in time roughlyÕ(m A ), where m A is the number of non-zero entries of A. This uses, in a non-trivial way, the breakthrough result of Spielman and Teng on inverting symmetric and diagonally-dominant matrices inÕ(m A ) time. Finally, we prove that e −x can be uniformly approximated up to a small additive error, in a non-negative interval [a, b] with a polynomial of degree roughly √ b − a. While this result is of independent interest in approximation theory, we show that, via the Lanczos method from numerical analysis, it yields a simple algorithm to compute exp(−A)v for symmetric PSD matrices that runs in time roughly O(t A · A ), where t A is time required for the computation of the vector Aw for given vector w. As an application, we obtain a simple and practical algorithm, with output conductance O( √ γ), for balanced separator that runs in timeÕ( m / √ γ). This latter algorithm matches the running time, but improves on the approximation guarantee of the Evolving-Sets-based algorithm by Andersen and Peres for balanced separator.

We present algorithms for solving a large class of flow and regression problems on unit weighted graphs to (1 + 1/poly(n)) accuracy in almost-linear time. These problems include ℓ p -norm minimizing flow for p large (p ∈ [ω(1), o(log 2/3 n)]), and their duals, ℓ p -norm semisupervised learning for p close to 1.As p tends to infinity, ℓ p -norm flow and its dual tend to max-flow and min-cut respectively. Using this connection and our algorithms, we give an alternate approach for approximating undirected max-flow, and the first almost-linear time approximations of discretizations of total variation minimization objectives.This algorithm demonstrates that many tools previous viewed as limited to linear systems are in fact applicable to a much wider range of convex objectives. It is based on the the routing-based solver for Laplacian linear systems by Spielman and Teng (STOC '04, SIMAX '14), but require several new tools: adaptive non-linear preconditioning, tree-routing based ultra-sparsification for mixed ℓ 2 and ℓ p norm objectives, and decomposing graphs into uniform expanders.

We introduce the sparsified Cholesky and sparsified multigrid algorithms for solving systems of linear equations. These algorithms accelerate Gaussian elimination by sparsifying the nonzero matrix entries created by the elimination process.We use these new algorithms to derive the first nearly linear time algorithms for solving systems of equations in connection Laplacians-a generalization of Laplacian matrices that arise in many problems in image and signal processing.We also prove that every connection Laplacian has a linear sized approximate inverse. This is an LU factorization with a linear number of nonzero entries that is a strong approximation of the original matrix. Using such a factorization one can solve systems of equations in a connection Laplacian in linear time. Such a factorization was unknown even for ordinary graph Laplacians. * This paper incorporates and improves upon results previously announced by a subset of the authors in [LPS15]. Fact 2.3. If A ≈ ǫ B and B ≈ δ C , then A ≈ ǫ+δ C .We now address one technicality of dealing with bDD matrices: it is not immediate whether or not a bDD matrix is singular. Moreover, if it is singular, the structure of its null space is not immediately clear either. Throughout the rest of this paper, we will consider bDD matrices to which a small multiple of the identity have been added. These matrices will be nonsingular. To reduce the problem of solving equations in a general bDD matrix M to that of solving equations in a nonsingular matrix, we require an estimate of the smallest nonzero eigenvalue of M .Claim 2.4. Suppose that all nonzero eigenvalues of M are at least µ and Z ≈ ǫ (M + ǫµI) −1 for some 0 < ǫ < 1/2. Given any b such that M x = b for some x , we have

We present a new algorithm for Independent Component Analysis (ICA) which has provable performance guarantees. In particular, suppose we are given samples of the form y = Ax + η where A is an unknown n × n matrix and x is a random variable whose components are independent and have a fourth moment strictly less than that of a standard Gaussian random variable and η is an n-dimensional Gaussian random variable with unknown covariance Σ: We give an algorithm that provable recovers A and Σ up to an additive ǫ and whose running time and sample complexity are polynomial in n and 1/ǫ. To accomplish this, we introduce a novel "quasi-whitening" step that may be useful in other contexts in which the covariance of Gaussian noise is not known in advance. We also give a general framework for finding all local optima of a function (given an oracle for approximately finding just one) and this is a crucial step in our algorithm, one that has been overlooked in previous attempts, and allows us to control the accumulation of error when we find the columns of A one by one via local search.

This monograph presents ideas and techniques from approximation theory for approximating functions such as x s , x −1 and e −x , and demonstrates how these results play a crucial role in the design of fast algorithms for problems which are increasingly relevant. The key lies in the fact that such results imply faster ways to compute primitives such as A s v, A −1 v, exp(−A)v, eigenvalues, and eigenvectors, which are fundamental to many spectral algorithms. Indeed, many fast algorithms reduce to the computation of such primitives, which have proved useful for speeding up several fundamental computations such as random walk simulation, graph partitioning, and solving systems of linear equations.

A community in a social network is usually understood to be a group of nodes more densely connected with each other than with the rest of the network. This is an important concept in most domains where networks arise: social, technological, biological, etc. For many years algorithms for finding communities implicitly assumed communities are nonoverlapping (leading to use of clustering-based approaches) but there is increasing interest in finding overlapping communities. A barrier to finding communities is that the solution concept is often defined in terms of an NP-complete problem such as Clique or Hierarchical Clustering.This paper seeks to initiate a rigorous approach to the problem of finding overlapping communities, where "rigorous" means that we clearly state the following: (a) the object sought by our algorithm (b) the assumptions about the underlying network (c) the (worst-case) running time.Our assumptions about the network lie between worst-case and average-case. An averagecase analysis would require a precise probabilistic model of the network, on which there is currently no consensus. However, some plausible assumptions about network parameters can be gleaned from a long body of work in the sociology community spanning five decades focusing on the study of individual communities and ego-centric networks (in graph theoretic terms, this is the subgraph induced on a node's neighborhood). Thus our assumptions are somewhat "local" in nature. Nevertheless they suffice to permit a rigorous analysis of running time of algorithms that recover global structure.Our algorithms use random sampling similar to that in property testing and algorithms for dense graphs. We note however that our networks are not necessarily dense graphs, not even in local neighborhoods.Our algorithms explore a local-global relationship between ego-centric and socio-centric networks that we hope will provide a fruitful framework for future work both in computer science and sociology.