An Agenda for Open Science in Communication

Abstract

In the last 10 years, many canonical findings in the social sciences appear unreliable. This so-called “replication crisis” has spurred calls for open science practices, which aim to increase the reproducibility, replicability, and generalizability of findings. Communication research is subject to many of the same challenges that have caused low replicability in other fields. As a result, we propose an agenda for adopting open science practices in Communication, which includes the following seven suggestions: (1) publish materials, data, and code; (2) preregister studies and submit registered reports; (3) conduct replications; (4) collaborate; (5) foster open science skills; (6) implement Transparency and Openness Promotion Guidelines; and (7) incentivize open science practices. Although in our agenda we focus mostly on quantitative research, we also reflect on open science practices relevant to qualitative research. We conclude by discussing potential objections and concerns associated with open science practices.

Digital Wellbeing Applications: Adoption, Use and Perceived Effects

“…In accordance with recent calls for greater transparency in Communication research (Dienlin et al, 2020), following IRB approval, prior to data collection, we pre-registered our research questions, hypotheses, and methodology for both the quantitative (https://osf.io/n3c2p?view_only=357262128e774814a83fb79300939428) and, acknowledging recent debates about the pre-registration of qualitative methods (Haven & Van Grootel, 2019), the qualitative (https://osf.io/kazdt?view_only=2139551dc24e4a17804264196f7da659) components of our study. Additionally, all materials, quantitative data, and analysis scripts are available through the OSF (https://osf.io/9ry6t/?view_only=a4a062bb14734545ab73c913c37a298b).…”

“…To ensure that our analyses were adequately powered, we conducted an a priori power analysis using G*Power (Faul et al, 2009). Given the generally small effect sizes observed in Communication research (Dienlin et al, 2020), we planned to be able to detect at least an effect of f 2 = .05 with 80% power and α = .05. For a multiple regression analysis with four predictors, we required a minimum sample size of n = 244.…”

Quantitative/Qualitative Methoden, Mixed methods

“…Je besser der Zugang zu qualitativ und quantitativ erhobenen Daten ist, desto eher sind nachträgliche Methodenkombinationen möglich, desto eher können Befunde repliziert bzw. Gründe für ein Scheitern der Replikation gesucht werden (Dienlin et al 2020;Medjedović 2019). Hier ist die Situation in der Politikwissenschaft mit den umfassenden Datenbeständen bei der GESIS positiver einzuschätzen als in der Kommunikationswissenschaft (Korcik 2017).…”

“…Dabei sollte zunächst die methodische Umsetzung im Mittelpunkt stehen. Divergierende Befunde fordern die Prüfung methodischer Designs auf Fehlerquellen sowie die Replizierbarkeit der Resultate (Dienlin et al 2020) heraus. Können derartige Fehlerquellen ausgeschlossen werden, können im folgenden Schritt die theoretischen Annahmen hinterfragt und spezifiziert werden.…”

Major socio-cultural barriers to widespread adoption of open scholarship

“…We propose that these areas could provide a high-level strategic basis for future research into the socio-cultural limitations of open scholarship, and could be transferred into our understanding of the different elements of open society (19). It will be most beneficial when different research communities can specify how the pluralistic values, principles, norms and practices of 'openness' can be embedded with minimal friction into their disciplines (20). Rather than simply acknowledging that 'open' scholarship is akin to 'better' scholarship, the ultimate aim here is to develop an inherently strategic approach to precisely when and where openness will be most effectively implemented in order to This is a non-peer-reviewed preprint achieve a desired outcome.…”