Blind trust in unblinded observation in Ecology, Evolution, and Behavior

Abstract

We surveyed 492 recent studies in the fields of ecology, evolution, and behavior (EEB) to evaluate potential for observer bias and the need for blind experimentation in each study. While 248 articles included experiments that could have been influenced by observer bias, only 13.3% of these articles indicated that experiments were blinded. The use of blind observation therefore was either grossly underreported in the surveyed articles, or many EEB studies were not blinded. We hope that a concerted effort of the field of EEB-including researchers, peer-reviewers, and journal editors-will help promote and institute routine, blind observation as an essential standard that should be practiced by all sciences.

Opinion of applied ethologists on expectation bias, blinding observers and other debiasing techniques

“…Probably, larger and more sustained concerted initiatives will be needed to raise the awareness of this scientific discipline in the long-term. These may include, for example, an increased emphasis on expectancy effects in textbooks for and the training of applied ethologists, further illustrations of the existence and consequences of expectation bias in ethological research outcomes, and critical evaluations of the potential of such bias in the design of experiments, in the assessment of research proposals and in the reviewing process of publication of research papers (Kardish et al, 2015).…”

“…It is surprising, therefore, that only a minority of the published animal behaviour studies report to have avoided expectation bias by blinding outcome assessors (Burghardt et al, 2012;Kardish et al, 2015;von Wilgenburh and Elgar, 2014). The aforementioned limited awareness about expectancy effects (the so-called bias blind spot) could be part of the explanation.…”

“…The present paper focusses on a particular type of unconscious bias, namely expectation bias (also termed confirmation bias, cognitive bias, observer bias or context effects). Expectation bias is a common source of inaccuracy where outcome assessors observe or interpret their observation in a way that supports their expectations or preferred hypothesis (Kardish et al, 2015). As a relatively young scientific discipline that relies heavily on behavioural methods, applied ethology would appear to be at high risk of expectation bias.…”

“…Despite these predisposing factors, and despite growing experimental evidence of expectation bias when using ethological research methods (Bohlen et al 2014;Hanlon, 2004, 2007;Tuyttens et al, 2014;van Wilgenburg and Elgar, 2013), blind observation is either grossly underreported or underutilized in animal behaviour studies (Burghardt et al, 2012;Kardish et al, 2015). Yet, blinding outcome assessors is widely accepted as an important aspect of a good study design to avoid expectation bias, in particular when outcome measures are not clearly defined, hard to perceive and require human judgement, and when the assessor has an interest in the outcome of the study (Rosenthal, 1966;Savović et al, 2012;Schulz and Grimes, 2002).…”

Promoting Transparency in Evolutionary Biology and Ecology

“…For instance, where observers are not 'blinded' to treatment or expected outcome, their studies report larger effect sizes on average and a higher proportion of significant p values than in studies with blinding (van Wilgenburg and Elgar 2013;Holman et al 2015). Although blinding is not possible in all studies in ecology and evolution, it is unfortunately quite rare even when feasible (Kardish et al 2015).…”