Quality Assessment of Meta-Analyses on Soil Organic Carbon

Abstract

Abstract. Soil organic carbon (SOC) plays a vital role in the global carbon cycle and is a potential sink for carbon dioxide. Agricultural management practices can support carbon sequestration and therefore offer potential removal strategies, whilst improving overall soil quality. Meta-analysis allows to summarize results from primary articles by calculating an overall effect size and hence to reveal the source of variation across studies. The number of meta-analyses published in the field of agriculture is continuously rising. At the same time, more and more articles refer to their synthesis work as a meta-analysis, despite applying less than rigorous methodologies. As a result, poor quality meta-analyses are published, which may lead to questionable conclusions and recommendations to scientists, policymakers and farmers. This study aims at quantitatively analyzing 31 meta-analyses, published between the years 2005–2020, studying the effects of different management practices on SOC. We compiled a quality criteria-set, suitable for soil and agricultural sciences, by adapting existing meta-analytical guidelines from other disciplines. The set is supported by a scoring scheme, which allows a quantitative analysis. The retrieved meta-analyses were structured according to 11 management categories, such as tillage, cover crops, residue management, biochar application etc., which allowed us to assess the state-of-knowledge on these categories. Major deficiencies were found in the use of standard metrics for effect size calculation, independence of effect sizes, standard deviation extraction for each study and weighting by the inverse of variance. Only one out of 31 SOC meta-analyses, which studied the effects of no-till/reduced tillage compared to conventional tillage, was found to be of high quality. Therefore, improved meta-analyses on the effects of e.g., organic agriculture, biochar, fertilization or crop diversification on SOC are urgently needed. We conclude that, despite the efforts over the last 15 years, the quality of meta-analyses on SOC research is still low. In order for the scientific community to provide high quality synthesis work and to make advancements in the sustainable management of agricultural soils, we need to adapt rigorous methodologies of meta-analysis as quickly as possible.