Best research practices in psychology: Illustrating epistemological and pragmatic considerations with the case of relationship science.

Abstract

In recent years, a robust movement has emerged within psychology to increase the evidentiary value of our science. This movement, which has analogs throughout the empirical sciences, is broad and diverse, but its primary emphasis has been on the reduction of statistical false positives. The present article addresses epistemological and pragmatic issues that we, as a field, must consider as we seek to maximize the scientific value of this movement. Regarding epistemology, this article contrasts the false-positives-reduction (FPR) approach with an alternative, the error balance (EB) approach, which argues that any serious consideration of optimal scientific practice must contend simultaneously with both false-positive and false-negative errors. Regarding pragmatics, the movement has devoted a great deal of attention to issues that frequently arise in laboratory experiments and one-shot survey studies, but it has devoted less attention to issues that frequently arise in intensive and/or longitudinal studies. We illustrate these epistemological and pragmatic considerations with the case of relationship science, one of the many research domains that frequently employ intensive and/or longitudinal methods. Specifically, we examine 6 research prescriptions that can help to reduce false-positive rates: preregistration, prepublication sharing of materials, postpublication sharing of data, close replication, avoiding piecemeal publication, and increasing sample size. For each, we offer concrete guidance not only regarding how researchers can improve their research practices and balance the risk of false-positive and false-negative errors, but also how the movement can capitalize upon insights from research practices within relationship science to make the movement stronger and more inclusive.

Benefits of open and high-powered research outweigh costs.

“…In this sense, we agree with Finkel et al (2015) that "research practices in our field will be better -in terms of scientific discovery and validity -in 2020 than they were in 2010" (p. 294). Ultimately, these discussions will improve research practices in psychology and consequently accelerate theoretical progress in our quest to understand human behavior.…”

Effects of tDCS on motor learning and memory formation: A consensus and critical position paper

“…Present problems in the field include: (1) insufficient use of double-blind designs (see above, for example only 25 out of the 60 published studies on tDCS effects on motor learning in healthy adults reviewed here utilized double-blind designs) and positive controls (stimulation of other cortical regions); (2) insufficient differentiation and understanding of design and claims when carrying out exploratory (hypothesis-generating) versus confirmatory (hypothesis-driven) research (the former suggesting trends and providing data for prospective power analysis and the latter, strengthened by preregistration (Finkel, Eastwick, & Reis, 2015), allowing drawing conclusions on particular effects; (3) insufficient efforts to reduce false-positive rates in studies geared to provide proof of principle data to power subsequent clinical trials; (4) scarcity of preregistration of hypothesis, design, power analysis and data processing for research written up as hypothesis-driven and confirmatory (see for example https://blogs.royalsociety.org/publishing/registered-reports/); (5) insufficient prepublication and sharing of materials (Lauer, Krumholz, & Topol, 2015;Morey et al, 2016), particularly in relation to negative results; (6) insufficient post-publication repositories of data (see for example (Campbell et al, 2002)) and in general (Nosek et al, 2015)) to allow additional analyses; (7) seldom use of experimental designs with replications built in (Anderson et al, 2016;Cohen et al, 1997;Gilbert, King, Pettigrew, & Wilson, 2016;Nosek et al, 2015); and (8) use of appropriate sample size based on prospective power analysis for studies claimed to be hypothesis-driven.…”

Solidarity through shared disadvantage: Highlighting shared experiences of discrimination improves relations between stigmatized groups.

“…Researchers studying the experiences of socioculturally devalued minority groups, however, seek populations that are, by definition, a minority of the population. Studying such groups can pose challenges, including: smaller recruitment pools; larger financial incentives for participation; and greater difficulty, expense, and time required to identify eligible participants (see also Finkel, Eastwick, & Reis, 2015).…”