Four Bad Habits of Modern Psychologists

Abstract

Four data sets from studies included in the Reproducibility Project were re-analyzed to demonstrate a number of flawed research practices (i.e., “bad habits”) of modern psychology. Three of the four studies were successfully replicated, but re-analysis showed that in one study most of the participants responded in a manner inconsistent with the researchers’ theoretical model. In the second study, the replicated effect was shown to be an experimental confound, and in the third study the replicated statistical effect was shown to be entirely trivial. The fourth study was an unsuccessful replication, yet re-analysis of the data showed that questioning the common assumptions of modern psychological measurement can lead to novel techniques of data analysis and potentially interesting findings missed by traditional methods of analysis. Considered together, these new analyses show that while it is true replication is a key feature of science, causal inference, modeling, and measurement are equally important and perhaps more fundamental to obtaining truly scientific knowledge of the natural world. It would therefore be prudent for psychologists to confront the limitations and flaws in their current analytical methods and research practices.

Small is beautiful: In defense of the small-N design

“…Recent articles by Grice et al, ( 2017 ) and Normand ( 2016 ) make similar points to those we make here. Like us, Normand stresses the importance of repeated measures on single individuals as a reliable and in many ways, preferable, approach for understanding psychological phenomena.…”

“…The fact that mathematical models most often predict performance at the individual participant level does not automatically preclude aggregation, by averaging or some other means, across participants, but does imply that conclusions made at the group level ought to be verified at the individual participant level (Grice et al, 2017 ). This implies that a sufficient number of trials must be collected at the individual level in order for such verification to be meaningful.…”

Beyond p values: utilizing multiple methods to evaluate evidence

“…Switching to this philosophy allows for techniques that match the daily activities of social scientists in their endeavors to unravel the story of how humans operate. OOM pushes the researcher to seek an inference to best explanation (Grice et al 2017). This causal inference procedure differs from both NHST and Bayes, where a researcher focuses on inferences to population parameters and their various assumptions underlying statistical tests (e.g., random sampling, normality, homogeneity of population treatment differences, etc.…”

Santé Psychique, Mal-Être, Dépression Et Anxiété Des Étudiants : Des Chiffres Et Des Humains?

“…Cette position narrative est d'autant plus pertinente si l'on considère la définition de Fernando Gil, rappelée par Bruno Falissard, de ce qu'est une preuve : « Une proposition est dite prouvée si, ayant été établie par une méthode reconnue, elle fait l'objet d'une croyance » (Falissard, 2005 (Grice et al, 2017;Munafò et al, 2017); ou plutôt son degré de réfutabilité comme le rappelle également Bruno Falissard : « on retiendra ici l'approche poppérienne, pour laquelle la méthode scientifique repose sur la formulation d'hypothèses réfutables par des expériences reproductibles » (Falissard, 2005 (Nuzzo, 2015). Il est donc également possible de considérer que les enquêtes peuvent en surface chercher à démontrer des faits de manière scientifique, mais qu'en réalité elles construisent des « mythes » ou des « éléments de narration » utiles potentiellement pour gagner l'opinion publique à une « cause spécifique » ou encore pour influencer les décideurs politiques dans un sens précis (Jorm, 2006).…”