Sensitivity of estimates of the effectiveness of REDD+ projects to matching specifications and moving from pixels to polygons as the unit of analysis

Abstract

There is a substantial interest in the potential of carbon credits generated by Reducing Emissions from tropical Deforestation and Degradation (REDD+) and traded on the voluntary carbon market for generating the finance needed to slow forest loss. However, such credits have become marred in controversy. Recent global-scale analysis using a range of methods for estimating the counterfactual rate of deforestation ex post suggest that many REDD+ projects have overestimated their effectiveness at reducing deforestation and consequently issued more credits than can be justified. All such methods include potentially arbitrary choices which can affect the estimate of the treatment effect. In addition, using pixels as the sampling unit, as some of the studies do, can introduce biases. One study which has been widely cited in the debate (Guizar-Coutino et al. 2022) estimated avoided deforestation using statistical matching of pixels and a single set of matching options. We estimate avoided deforestation from the same set of projects using 7-hectare plots rather than pixels to sample deforestation and explore the sensitivity of the results to matching choices (exploring 120 matched sets in total). We filtered the results on three criteria: 1) post-matching covariate balance, 2) proportion of REDD+ samples that were successfully matched, and 3) similarity of trends in deforestation rates prior to REDD+ implementation (parallel trends). While one of the 44 REDD+ projects failed these quality control process, we estimate treatment effects for the remaining 43 projects. There was a substantial correlation between our new estimates and those published in 2022 (0.72 measured in annual percent change, and 0.9 measured in total area change) and our headline estimate of 0.22% per yr (95% CI: 0.13 to 0.34) is essentially unchanged. At a time when confidence in the voluntary carbon markets is low, we hope these results provide reassurance that ex-post counterfactual estimates of avoided deforestation are consistent, helping accelerate their widespread adoption and rebuild trust in nature-based climate solutions.