Conceptions of Good Science in Our Data-Rich World

Abstract

Scientists have been debating for centuries the nature of proper scientific methods. Currently, criticisms being thrown at data-intensive science are reinvigorating these debates. However, many of these criticisms represent long-standing conflicts over the role of hypothesis testing in science and not just a dispute about the amount of data used. Here, we show that an iterative account of scientific methods developed by historians and philosophers of science can help make sense of data-intensive scientific practices and suggest more effective ways to evaluate this research. We use case studies of Darwin's research on evolution by natural selection and modern-day research on macrosystems ecology to illustrate this account of scientific methods and the innovative approaches to scientific evaluation that it encourages. We point out recent changes in the spheres of science funding, publishing, and education that reflect this richer account of scientific practice, and we propose additional reforms.

Data-Intensive Science and Research Integrity

“…Third, during the data-interpretation stage researchers make variousassumptions based on interpretive frameworks, analytical tools, concepts, and background theories (Elliott et al 2016, Longino 1990). Deception could occur if researchers overstate the scientific or practical significance of their findings or claim to have demonstrated causal connections when they have only produced evidence of statistical associations (Resnik 2000, Master and Resnik 2013).…”

“…“big data”) is one of the most significant trends in contemporary research (Bell et al 2009, Miller 2010, Elliott et al 2016, Sigumoto et al 2016). Data-intensive science has been described as research in which the capture, curation, and analysis of large volumes of data is central to the scientific question.…”

“…Nevertheless, contemporary data-intensive science is distinctive because it incorporates new computational methods and technologies and tends to involve large, interdisciplinary scientific teams (Leonelli 2014; Strasser 2012). Data-intensive research now plays an important role in many disciplines, including particle physics, astronomy, cancer therapy, drug discovery, radiology, ecology, climatology, geology, molecular biology, molecular biology, systems biology, economics, and social science (Elliott et al 2016, Sugimoto et al 2016). …”

“…In other cases, research which is initially designed to confirm or falsify a hypothesis may yield large quantities of data that may be reused for more data-driven research of an exploratory nature, leading to thediscovery of unexpected patterns or associations. On this view, data-intensive and hypothesis-driven modes of research both play important roles in scientific investigation(Elliott et al 2016). …”

“…For example, computational researchers have reverse engineered computer-generated algorithms to peer inside the “black box” of computer-mediated statistical inference. Data-intensive methods may also be treated as components of broader research programs in which statistical associations are subjected to further investigation (Kitchin 2014, Leonelli 2012) Elliott et al (2016). argue that instead of privileging one form of inquiry over another, scientists should focus on employing the best methods for addressing knowledge gaps and creating research teams with the necessary expertise to employ those methods successfully.…”