Let M be a complete non-compact manifold satisfying the volume doubling condition, with doubling index N and reverse doubling index n, n ≤ N, both for large balls. Assume a Gaussian upper bound for the heat kernel, and an L 2 -Poincaré inequality outside a compact set. If 2 < n, then we show that for p ∈ (2, n), (R p ): L p -boundedness of the Riesz transform, (G p ): L p -boundedness of the gradient of the heat semigroup, and (RH p ): reverse L p -Hölder inequality for the gradient of harmonic functions, are equivalent to each other. Our characterization implies that for p ∈ (2, n), (R p ) has an open ended property and is stable under gluing operations. This substantially extends the well known equivalence of (R p ) and (G p ) from [4] to more general settings, and is optimal in the sense that (R p ) does not hold for any p ≥ n > 2 on manifolds having at least two Euclidean ends of dimension n. For p ∈ (max{N, 2}, ∞), the fact that (R p ), (G p ) and (RH p ) are equivalent essentially follows from [22]; moreover, if M is non-parabolic, then any of these conditions implies that M has only one end. For the proof, we develop a new criteria for boundedness of the Riesz transform, which was nontrivially adapted from [4], and make an essential application of results from [22]. Our result allows extensions to non-smooth settings.