Transparency in Ecology and Evolution: Real Problems, Real Solutions

Parker, Timothy; Forstmeier, Wolfgang; Koricheva, Julia; Fidler, Fiona; Hadfield, Jarrod D.; Chee, Yung En; Kelly, Clint D.; Gurevitch, Jessica; Nakagawa, Shinichi

doi:10.1016/j.tree.2016.07.002

Cited by 170 publications

(188 citation statements)

References 57 publications

Supporting

Mentioning

177

Contrasting

Unclassified

Order By: Relevance

“…The situation may get worse. In many studies, sample sizes are low, resulting in statistical power that is often as low as 20% Smith, Hardy & Gammell, 2011;Button et al, 2013;Parker et al, 2016). In this situation we will have 80 instead of 20 cases of false-negative results (black in Fig.…”

Section: Problems (1) the Argument Of Ioannidis And Some Extensionsmentioning

confidence: 99%

“…In the fields of ecology and evolution observed effect sizes are typically small (e.g. r = 0.19; , which is still likely an overestimate (Hereford, Hansen & Houle, 2004;Parker et al, 2016). Hence, large sample sizes are required to detect such effects (required N = 212, for detecting r = 0.19 with 80% power).…”

Section: Problems (1) the Argument Of Ioannidis And Some Extensionsmentioning

confidence: 99%

“…The existing literature is heavily biased towards stories of success (Parker et al, 2016), with 84% of all publications finding support for their initial hypotheses (Fanelli, 2010). As we will see in Section II.2, this figure is far from an objective representation of all hypothesis tests that have been conducted, because null findings (non-significant results) are less likely to get published (Rosenthal, 1979;Simonsohn et al, 2014), and because various common data-analysis practices increase the rate of false positives as well as the average strength of reported effects among those results that are published (Anderson, Martinson & De Vries, 2007;Simmons et al, 2011;John, Loewenstein & Prelec, 2012;Parker et al, 2016). Even without problematic data analysis, some (false) positive evidence can emerge for any hypothesis (Fig.…”

Section: Problems (1) the Argument Of Ioannidis And Some Extensionsmentioning

confidence: 99%

“…Several research fields appear to be in crisis of confidence (McNutt, 2014;Nuzzo, 2014Nuzzo, , 2015Horton, 2015;Parker et al, 2016) as evidence emerges that the majority of published research findings cannot be replicated (Ioannidis, 2005;Pereira & Ioannidis, 2011;Prinz, Schlange & Asadullah, 2011;Begley & Ellis, 2012;Open Science Collaboration, 2015). According to a recent survey in Nature (Baker, 2016), 52% of researchers believe that there is 'a significant crisis', 38% see 'a slight crisis', and only 3% see 'no crisis'.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Detecting and avoiding likely false‐positive findings – a practical guide

2016

Self Cite

View full text Add to dashboard Cite

Recently there has been a growing concern that many published research findings do not hold up in attempts to replicate them. We argue that this problem may originate from a culture of 'you can publish if you found a significant effect'. This culture creates a systematic bias against the null hypothesis which renders meta-analyses questionable and may even lead to a situation where hypotheses become difficult to falsify. In order to pinpoint the sources of error and possible solutions, we review current scientific practices with regard to their effect on the probability of drawing a false-positive conclusion. We explain why the proportion of published false-positive findings is expected to increase with (i) decreasing sample size, (ii) increasing pursuit of novelty, (iii) various forms of multiple testing and researcher flexibility, and (iv) incorrect P-values, especially due to unaccounted pseudoreplication, i.e. the non-independence of data points (clustered data). We provide examples showing how statistical pitfalls and psychological traps lead to conclusions that are biased and unreliable, and we show how these mistakes can be avoided. Ultimately, we hope to contribute to a culture of 'you can publish if your study is rigorous'. To this end, we highlight promising strategies towards making science more objective. Specifically, we enthusiastically encourage scientists to preregister their studies (including a priori hypotheses and complete analysis plans), to blind observers to treatment groups during data collection and analysis, and unconditionally to report all results. Also, we advocate reallocating some efforts away from seeking novelty and discovery and towards replicating important research findings of one's own and of others for the benefit of the scientific community as a whole. We believe these efforts will be aided by a shift in evaluation criteria away from the current system which values metrics of 'impact' almost exclusively and towards a system which explicitly values indices of scientific rigour.

show abstract

Section: Problems (1) the Argument Of Ioannidis And Some Extensionsmentioning

confidence: 99%

Section: Problems (1) the Argument Of Ioannidis And Some Extensionsmentioning

confidence: 99%

Section: Problems (1) the Argument Of Ioannidis And Some Extensionsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Detecting and avoiding likely false‐positive findings – a practical guide

2016

Self Cite

View full text Add to dashboard Cite

show abstract

“…Reproducibility of scientific findings is a cornerstone of the scientific method but concerns are mounting due to replications of published findings failing to yield similar results (Parker et al, 2016;Clark, 2017). One issue inherent to working with crocodylians is small sample sizes due to space requirement, ethical limits and extensive safety requirements.…”

Section: Reproducibility and Statistical Powermentioning

confidence: 99%

DIVING IN A WARMING WORLD: Thermal constraints on the diving capacity of estuarine crocodiles (Crocodylus porosus)

Rodgers¹

View full text Add to dashboard Cite

A central challenge in conserving biodiversity is predicting the consequences of anthropogenic climate change on species' distributions and persistence. Forced climate change has severely altered thermal regimes in marine and freshwater habitats. Rapid escalations in environmental temperatures may be particularly threatening to ectothermic species (almost all plants, invertebrates, fish, amphibians and reptiles), where body temperature and concomitant functional performance are strongly tied to the thermal environment. The threat of overheating is salient for air-breathing, ectothermic divers, such as the estuarine crocodile (Crocodylus porosus, Schneider, 1801), because submergence times are inversely related to water temperature. It is unknown how C. porosus will fare in warming waters but diving oxygen stores are hypothesised to be consumed more rapidly at elevated temperatures leading to a reduction of aerobic dive limits (i.e. maximum submergence time before lactate is accumulated). Shorter dive durations may force animals to spend more time at the water surface, leaving less time available for obligate underwater activities (e.g. predator avoidance and hunting for aquatic prey).This thesis assessed the effect of elevated water temperatures (emulating climate change scenarios) on the diving physiology and behaviour of C. porosus. The thermal sensitivity of predator avoidance dives (i.e. minutes submerged) was assessed in juveniles at three water temperatures reflecting climate change scenarios (Chapter 2). Diving performance was thermally sensitive with dive durations halving between the 'no warming' and 'moderate warming' scenarios. Ectotherms are however revered for their thermal acclimation/acclimatisation capacity following long term exposure to novel temperatures; whereby an animal's underlying physiology is responsively altered to maintain or optimise performance. For this reason, the acclimation capacity of C. porosus was assessed by exposing crocodiles to thermal acclimation treatments for a minimum of 30 days. Thermal acclimation treatments had no effect on dive durations -a result indicative of absent thermal acclimation capacity at elevated temperatures within 30 days.ii The physiological mechanisms underlying compromised diving performance at elevated temperatures were subsequently examined (Chapter 3). Reduced diving performance was hypothesised to be linked to increased oxygen demands and a reduced capacity for metabolic depression at elevated temperatures. Diving oxygen uptake, diving heart rate and post-dive plasma-lactate concentrations were assessed at two test temperatures (i.e. 28•C and 34•C). Diving metabolic rate increased threefold between 28•C and 34•C and the capacity to depress metabolic demands (from surface levels) was inhibited by 46%. Post-dive plasma-lactate accumulation was independent of water temperature. Collectively, these results show the aerobic dive limit of C. porosus was significantly reduced at elevated temperatures and animals behaviourally terminated dives...

show abstract

Ten principles for generating accessible and useable COVID‐19 environmental science and a fit‐for‐purpose evidence base

Kadykalo

Haddaway

Rytwinski

et al. 2021

Ecol Sol and Evidence

View full text Add to dashboard Cite

1. The 'anthropause' , a period of unusually reduced human activity and mobility due to COVID-19 restrictions, has serendipitously opened up unique opportunities for research on how human activities impact the environment. 2. In the field of health, COVID-19 research has led to concerns about the quality of research papers and the underlying research and publication processes due to accelerated peer review and publication schedules, increases in pre-prints and retractions. 3. In the field of environmental science, framing the pandemic and associated global lockdowns as an unplanned global human confinement experiment with urgency should raise the same concerns about the rigorousness and integrity of the scientific process. Furthermore, the recognition of an 'infodemic' , an unprecedented explosion of research, risks research waste and duplication of effort, although how information is used is as important as the quality of evidence. This highlights the need for an evidence base that is easy to find and use-that is discoverable, curated, synthesizable, synthesized. 4. We put forward a list of 10 key principles to support the establishment of a reproducible, replicable, robust, rigorous, timely and synthesizable COVID-19 environmental evidence base that avoids research waste and is resilient to the pressures to publish urgently. These principles focus on engaging relevant actors (e.g. local communities, rightsholders) in research design and production, statistical power, collaborations, evidence synthesis, research registries and protocols, open science and transparency, data hygiene (cleanliness) and integrity, peer review transparency, standardized keywords and controlled vocabularies.

show abstract

Transparency in Ecology and Evolution: Real Problems, Real Solutions

Cited by 170 publications

References 57 publications

Detecting and avoiding likely false‐positive findings – a practical guide

Detecting and avoiding likely false‐positive findings – a practical guide

DIVING IN A WARMING WORLD: Thermal constraints on the diving capacity of estuarine crocodiles (Crocodylus porosus)

Ten principles for generating accessible and useable COVID‐19 environmental science and a fit‐for‐purpose evidence base

Contact Info

Product

Resources

About