Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences

Christie, Alec P.; Abecasis, David; Adjeroud, Mehdi; Alonso, Juan Carlos; Amano, Tatsuya; Antón, Álvaro; Baldigo, Barry P.; Barrientos, Rafael; Bicknell, Jake E.; Buhl, Deborah A.; Cebrián, Just; Ceia, Ricardo S.; Cibils‐Martina, Luciana; Clarke, Sarah; Claudet, Joachim; Craig, Michael; Davoult, Dominique; Backer, Annelies De; Donovan, Mary K.; Eddy, Tyler D.; França, Filipe; Gardner, Jonathan P. A.; Harris, Bradley P.; Huusko, Ari; Jones, Ian L.; Kelaher, Brendan P.; Kotiaho, Janne S.; López‐Baucells, Adrià; Major, Heather L.; Mäki‐Petäys, Aki; Martín, Beatriz; Martín, Carlos A.; Martin, Philip A.; Mateos-Molina, Daniel; McConnaughey, Robert A.; Meroni, Michele; Meyer, Christoph F. J.; Mills, Kade; Montefalcone, Monica; Noreika, Norbertas; Palacı́n, Carlos; Pande, Anjali; Pitcher, C. Roland; Ponce, Carlos; Rinella, Matthew J.; Rocha, Ricardo; Ruiz-Delgado, María C.; Schmitter‐Soto, Juan J.; Shaffer, Jill A.; Sharma, Shailesh; Sher, Anna A.; Stagnol, Doriane; Stanley, Thomas R.; Stokesbury, Kevin D. E.; Torres, Aurora; Tully, Oliver; Vehanen, Teppo; Watts, Corinne; Zhao, Qingyuan; Sutherland, William J.

doi:10.1038/s41467-020-20142-y

Cited by 72 publications

(86 citation statements)

References 71 publications

(52 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The other perception, that non-English-language studies tend to adopt less robust designs, seems to be supported by our results, although a reasonable number of non-English-language studies with robust designs also exist, especially in Portuguese (25 studies with Randomised Controlled Trial) and Spanish (13 studies with Before-After-Control-Impact and three with Randomised Controlled Trial). Scientific evidence presented in non-English-language studies could thus be lower in quality, and suffer from more serious biases, on average, compared to that provided by English-language studies [24]. This difference in evidence quality between English-language and non-English language studies is likely to create a trade-off in evidence-poor regions, between the availability of context-specific evidence and the quality of evidence; for some species and locations, the only available evidence might be found in non-English-language studies based on less robust designs [25].…”

Section: Discussionmentioning

confidence: 99%

“…To test whether there was a difference in study designs adopted between studies in different languages, we only included studies based on one of the following five designs: After, Before-After (BA), Control-Impact (CI), Before-After-Control-Impact (BACI), and Randomised Controlled Trial (RCT). These study designs were recorded as an ordinal variable with RCT being the least biased design, followed by BACI, CI, BA, and After, based on results from [24]. Considering that English-language studies in English-speaking countries (especially the UK and the US) may adopt more robust study designs than English-language studies in other countries, English-language studies were further divided into two groups; studies conducted in countries where English is an official language (“English – official”), and studies in all other countries (“English – others”), using information on countries’ official languages in [40].…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Tapping into non-English-language science for the conservation of global biodiversity

Amano

Berdejo‐Espinola

Christie

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The widely held assumption that any important scientific information would be available in English underlies the underuse of non-English-language science across disciplines. However, non-English-language science is expected to bring unique and valuable scientific information, especially in disciplines where the evidence is patchy, and for emergent issues where synthesising available evidence is an urgent challenge. Yet such contribution of non-English-language science to scientific communities and the application of science is rarely quantified. Here we show that non-English-language studies provide crucial evidence for informing global biodiversity conservation. By screening 419,680 peer-reviewed papers in 16 languages, we identified 1,234 non-English-language studies providing evidence on the effectiveness of biodiversity conservation interventions, compared to 4,412 English-language studies identified with the same criteria. Relevant non-English-language studies are being published at an increasing rate, and can expand the geographical (by 12-25%) and taxonomic (by 5-32%) coverage of English-language evidence, especially in biodiverse regions, albeit often based on less robust study designs. Our results show that synthesising non-English-language studies is key to overcoming the widespread lack of local, context-dependent evidence and facilitating evidence-based conservation globally. We urge wider disciplines to rigorously reassess the untapped potential of non-English-language science in informing decisions to address other global challenges.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Tapping into non-English-language science for the conservation of global biodiversity

Amano

Berdejo‐Espinola

Christie

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The first knowledge gap is spatial: there is limited information from less seasonal regions (figure 1a), including critical areas under threat of forest degradation such as the Purus-Madeira interfluve, which links the central and southeastern Amazon [34]. The second gap is temporal: few studies show temporal trajectories of changes in forest functioning after fire [9,20,24] despite this being crucial to quantify impacts with less bias [35]. The third gap is in understanding to what extent plant morphological traits can avoid the effects of fire intensity on post-fire mortality [21][22][23] and post-fire recovery [24].…”

Section: Introductionmentioning

confidence: 99%

Drought-driven wildfire impacts on structure and dynamics in a wet Central Amazonian forest

et al. 2021

View full text Add to dashboard Cite

While the climate and human-induced forest degradation is increasing in the Amazon, fire impacts on forest dynamics remain understudied in the wetter regions of the basin, which are susceptible to large wildfires only during extreme droughts. To address this gap, we installed burned and unburned plots immediately after a wildfire in the northern Purus-Madeira (Central Amazon) during the 2015 El-Niño. We measured all individuals with diameter of 10 cm or more at breast height and conducted recensuses to track the demographic drivers of biomass change over 3 years. We also assessed how stem-level growth and mortality were influenced by fire intensity (proxied by char height) and tree morphological traits (size and wood density). Overall, the burned forest lost 27.3% of stem density and 12.8% of biomass, concentrated in small and medium trees. Mortality drove these losses in the first 2 years and recruitment decreased in the third year. The fire increased growth in lower wood density and larger sized trees, while char height had transitory strong effects increasing tree mortality. Our findings suggest that fire impacts are weaker in the wetter Amazon. Here, trees of greater sizes and higher wood densities may confer a margin of fire resistance; however, this may not extend to higher intensity fires arising from climate change.

show abstract

“…number of clusters). One possible workaround could be to iterate the clustering step (similar to VoPo [ 36 ]) and differential abundance analysis, whose results can then be combined to obtain regions in marker space of high probability of an association with the corresponding covariate.…”

Section: Discussionmentioning

confidence: 99%

Censcyt: censored covariates in differential abundance analysis in cytometry

Gerber

Robinson

2021

BMC Bioinformatics

View full text Add to dashboard Cite

Background Innovations in single cell technologies have lead to a flurry of datasets and computational tools to process and interpret them, including analyses of cell composition changes and transition in cell states. The diffcyt workflow for differential discovery in cytometry data consist of several steps, including preprocessing, cell population identification and differential testing for an association with a binary or continuous covariate. However, the commonly measured quantity of survival time in clinical studies often results in a censored covariate where classical differential testing is inapplicable. Results To overcome this limitation, multiple methods to directly include censored covariates in differential abundance analysis were examined with the use of simulation studies and a case study. Results show that multiple imputation based methods offer on-par performance with the Cox proportional hazards model in terms of sensitivity and error control, while offering flexibility to account for covariates. The tested methods are implemented in the package censcyt as an extension of diffcyt and are available at https://bioconductor.org/packages/censcyt. Conclusion Methods for the direct inclusion of a censored variable as a predictor in GLMMs are a valid alternative to classical survival analysis methods, such as the Cox proportional hazard model, while allowing for more flexibility in the differential analysis.

show abstract

Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences

Cited by 72 publications

References 71 publications

Tapping into non-English-language science for the conservation of global biodiversity

Tapping into non-English-language science for the conservation of global biodiversity

Drought-driven wildfire impacts on structure and dynamics in a wet Central Amazonian forest

Censcyt: censored covariates in differential abundance analysis in cytometry

Contact Info

Product

Resources

About