Research in Social Psychology Changed Between 2011 and 2016: Larger Sample Sizes, More Self-Report Measures, and More Online Studies

Abstract

The debate about false positives in psychological research has led to a demand for higher statistical power. To meet this demand, researchers need to collect data from larger samples—which is important to increase replicability, but can be costly in both time and money (i.e., remuneration of participants). Given that researchers might need to compensate for these higher costs, we hypothesized that larger sample sizes might have been accompanied by more frequent use of less costly research methods (i.e., online data collection and self-report measures). To test this idea, we analyzed social psychology studies published in 2009, 2011, 2016, and 2018. Indeed, research reported in 2016 and 2018 (vs. 2009 and 2011) had larger sample sizes and relied more on online data collection and self-report measures. Thus, over these years, research improved in its statistical power, but also changed with regard to the methods applied. Implications for social psychology as a discipline are discussed.

‘Personality in its Natural Habitat’ Revisited: A Pooled, Multi–sample Examination of the Relationships between the Big Five Personality Traits and Daily Behaviour and Language Use

“…At the same time, it can stand in comparison with the modern, technology‐enhanced, big(ger)‐data, mobile sensing personality studies and highlights their value and unique potentials (Harari et al, 2019; Montag & Elhai, 2019; Stachl et al, 2020). And, last but not least, as the need for replication is becoming clear and consensually accepted in the field (Zwaan, Etz, Lucas, & Donnellan, 2018), the project can also serve as a reminder to try to replicate not only easily reproduced research but also the more resource‐intensive and challenging‐to‐conduct studies (Sassenberg & Ditrich, 2019), so that psychology will over time accrue a solid knowledge base of replicated facts of the broadest possible range of its science.…”

Adversarial alignment enables competing models to engage in cooperative theory building toward cumulative science

“…Science also advances from efforts to understand how seemingly incompatible findings can be resolved, how more refined techniques allow deeper insight into mechanisms underlying classic findings, and how variations in research approaches, designs, or measures may uncover moderating factors that help establish the generality vs. specificity of prior observations (2). Yet, analyses of publication trends reveal that a focus on optimizing empirical techniques can have the side effect of reducing the range of theoretical issues under examination (3).…”

Using a Telegram chatbot as cost-effective software infrastructure for ambulatory assessment studies with iOS and Android devices

“…Although our approach is not a complete solution for AA studies (i.e., surveys must be prepared on a separate platform), its implementation is rather easy, as many researchers already have access to online-survey software (cf. Sassenberg & Ditrich, 2019). In addition, only limited resources are needed to create a chatbot account and to set up a computer/server as platform for our Python script as it can be edited under Windows, macOS, and Linux.…”