Assessment of the quality of meta-analysis in agronomy

Philibert, Aurore; Loyce, Chantal; Makowski, David

doi:10.1016/j.agee.2011.12.003

Cited by 189 publications

(191 citation statements)

References 33 publications

Supporting

Mentioning

185

Contrasting

Unclassified

Order By: Relevance

“…Ways by which authors of reviews can improve the reliability of their syntheses mainly relate to better reporting and have been detailed extensively elsewhere (e.g. Haddaway et al, 2015;Koricheva and Gurevitch, 2014;Philibert et al, 2012;Roberts et al, 2006). These improvements include aspects such as the development of an apriori protocol, reporting of search strategies and inclusion criteria, and detailing the included studies and extracted data.…”

Section: Resultsmentioning

confidence: 99%

“…Guyatt et al, 2011;Shea et al, 2009;Woodcock et al, 2014). Within environmental science, most evaluations to date have focused on reviews within specific disciplines and that apply meta-analytical techniques (Huntington, 2011;Koricheva and Gurevitch, 2014;Philibert et al, 2012), identifying consistent weaknesses in conduct and reporting standards.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The reliability of evidence review methodology in environmental science and conservation

O‘Leary

Kvist

Bayliss

et al. 2016

Environmental Science & Policy

View full text Add to dashboard Cite

Given the proliferation of primary research articles, the importance of reliable environmental evidence reviews for informing policy and management decisions is increasing. Although conducting reviews is an efficient method of synthesising the fragmented primary evidence base, reviews that are of poor methodological reliability have the potential to misinform by not accurately reflecting the available evidence base. To assess the current value of evidence reviews for decision-making we appraised a systematic sample of articles published in early 2015 (N=92) using the Collaboration for Environmental Evidence Synthesis Assessment Tool (CEESAT). CEESAT assesses the methodology of policy-relevant evidence reviews according to elements important for objectivity, transparency and comprehensiveness. Overall, reviews performed poorly with a median score of 2.5/39 and a modal score of zero (range 0-30, mean 5.8), and low scores were ubiquitous across subject areas. In general, reviews that applied meta-analytical techniques achieved higher scores than narrative syntheses (median 18.3 and 2.0 respectively), as a result of the latter consistently failing to adequately report methodology or how conclusions were drawn. However, some narrative syntheses achieved high scores, illustrating that the reliability of reviews should be assessed on a case-by-case basis. Given the potential importance of reviews for informing management and policy, as well as research, it is vital that overall methodological reliability is improved. Although the increasing number of systematic reviews and meta-analyses highlight that some progress is being made, our findings suggest little or no improvement in the last decade. To motivate progress, we recommend that an annual assessment of the methodological reliability of evidence reviews be conducted. To better serve the environmental policy and management communities we identify a requirement for independent critical appraisal of review methodology thus enabling decisionmakers to select reviews that are most likely to accurately reflect the evidence base.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

The reliability of evidence review methodology in environmental science and conservation

O‘Leary

Kvist

Bayliss

et al. 2016

Environmental Science & Policy

View full text Add to dashboard Cite

show abstract

“…We repeated the homogeneity test and mixed model regression analyses excluding all individual field and growth chamber studies one at a time as well as for a subset of the data excluding all of the modeling studies (Tudoreanu and Phillips 2004;Philibert et al 2012). This provided an indication of whether the dominant factors were still significant as the database changed.…”

Section: Period Of Nitrous Oxide Measurementmentioning

confidence: 99%

Do cover crops increase or decrease nitrous oxide emissions? A meta-analysis

Basche¹,

Miguez²,

Kaspar³

et al. 2014

Journal of Soil and Water Conservation

301

252

View full text Add to dashboard Cite

There are many environmental benefits to incorporating cover crops into crop rotations, such as their potential to decrease soil erosion, reduce nitrate (NO 3 ) leaching, and increase soil organic matter. Some of these benefits impact other agroecosystem processes, such as greenhouse gas emissions. In particular, there is not a consensus in the literature regarding the effect of cover crops on nitrous oxide (N 2 O) emissions. Compared to site-specific studies, meta-analysis can provide a more general investigation into these effects. Twenty-six peer-reviewed articles including 106 observations of cover crop effects on N 2 O emissions from the soil surface were analyzed according to their response ratio, the natural log of the N 2 O flux with a cover crop divided by the N 2 O flux without a cover crop (LRR). Forty percent of the observations had negative LRRs, indicating a cover crop treatment which decreased N 2 O, while 60% had positive LRRs indicating a cover crop treatment which increased N 2 O. There was a significant interaction between N rate and the type of cover crop where legumes had higher LRRs at lower N rates than nonlegume species. When cover crop residues were incorporated into the soil, LRRs were significantly higher than those where residue was not incorporated. Geographies with higher total precipitation and variability in precipitation tended to produce higher LRRs. Finally, data points measured during cover crop decomposition had large positive LRRs and were larger than those measured when the cover crop was alive. In contrast, those data points measuring for a full year had LRRs close to zero, indicating that there was a balance between periods when cover crops increased N 2 O and periods when cover crops decreased emissions. Therefore, N 2 O measurements over the entire year may be needed to determine the net effect of cover crops on N 2 O. The data included in this meta-analysis indicate some overarching crop management practices that reduce direct N 2 O emissions from the soil surface, such as no soil incorporation of residues and use of nonlegume cover crop species. However, our results demonstrate that cover crops do not always reduce direct N 2 O emissions from the soil surface in the short term and that more work is needed to understand the full global warming potential of cover crop management. Key words: cover crops-global warming potential-meta-analysis-nitrous oxideAgricultural soils account for 69% of nitrous oxide (N 2 O) emissions in the United States (USEPA 2013). This occurs because nitrogen (N) is an essential nutrient for agricultural production; N is added to soil as N fertilizer and manure, released from soil organic matter, and has high reactivity and mobility in terrestrial ecosystems (Robertson and Vitousek 2009). Fertilizer N recovery efficiency for major cereal production is less than 50% and even as low as 20% (Cassman et al. 2002), which potentially makes large quantities of N available for the biological processes that release N 2 O. Nitrous oxide, which ...

show abstract

“…The potential for publication bias was investigated and sensitivity analyses were conducted to examine how individual observations influenced weighted mean effect sizes (Philibert et al, 2012). Sampling variances were not available for individual observations, thus standard errors were approximated to estimate the precision of the effect size by randomly Fig.…”

Section: Data Extraction and Analysismentioning

confidence: 99%

First‐Season Crop Yield Response to Organic Soil Amendments: A Meta‐Analysis

et al. 2017

View full text Add to dashboard Cite

Core Ideas Organic amendments are promoted as sustainable alternatives to synthetic fertilizer. Crop yield increased by an average of 43% in the first season after organic soil amendment. Yield benefit from organic amendments was greater in leafy crops than root crops. Poultry manure was commonly used and provided the greatest agronomic benefit. Yield benefit of organic amendment was lower in arid regions with poor soil quality. Organic soil amendments are increasingly promoted as a sustainable alternative to synthetic fertilizers and as a tool for building soil quality through improved chemical, physical, and biological properties. However, short‐term yield response to organic amendments is highly variable. A meta‐analysis of 53 studies was conducted to (i) develop a global estimate of first‐season crop yield response to organic amendments, and (ii) determine the effect of crop type, amendment characteristics, soil properties, cultural practices, and climate on the magnitude of this yield response. Yield response ratios were calculated (organic amendment yield compared to a non‐fertilized control) and differences among groups were determined using 95% bootstrap confidence intervals (CI). Across all studies, crop yield increased 43±7% (95% CI) in the first‐season after an organic amendment. Yield response was greatest for leafy crops (71±26% increase) and lowest for root/tuber/bulb crops (29±10% increase). Poultry manure/compost was the most commonly used amendment and provided a yield increase of 76±21%. In contrast, plant‐based amendments increased yield by only 27±9%. Amendment application rate alone was not an effective predictor of yield response, and there were not enough studies available to explore the possible interaction between amendment type and rate. Yield benefits of organic amendments were muted in soils with high organic matter and in arid climates. These results help identify options for maximizing the agronomic value of organic amendments, and suggest research is needed to improve agronomic efficiency of amendments in arid regions with poor soil quality.

show abstract

Assessment of the quality of meta-analysis in agronomy

Cited by 189 publications

References 33 publications

The reliability of evidence review methodology in environmental science and conservation

The reliability of evidence review methodology in environmental science and conservation

Do cover crops increase or decrease nitrous oxide emissions? A meta-analysis

First‐Season Crop Yield Response to Organic Soil Amendments: A Meta‐Analysis

Contact Info

Product

Resources

About