The reliability and temporal stability of self-reported media exposure - a meta-analysis

Abstract

The measurement of media exposure is essential to not only traditional audience research, but also media effects research which relies on accurate estimates of media exposure. Even in the age of digital trace data and passive audience measurement, the workhorse of basically all communication research is self-report data. In this paper, I present a meta-analysis of the reliability and temporal stability of media exposure self-reports. Results show that media self-reported exposure was moderately reliable and highly stable. The estimated reliability was lower in youth samples, while rank-order stability was very similar for a adults and adolescents. Moderation analyses showed that exposure to specific outlets yielded more reliable information in adult samples, while media-specific differences in reliability were only found in youth samples.Keywords: meta-analysis, reliability, media exposure, panel dataThe measurement of media exposure is essential to not only traditional audience research, but also media effects research which relies on accurate estimates of media exposure. Even in the age of digital trace data and passive audience measurement, the workhorse of basically all communication research is self-report data. For several decades, scholars have investigated the reliability, accuracy and validity of self-reported media exposure measures (Allen, 1981;Chaffee & Schleuder, 1986), with most methodological research being conducted on question and response formats (Andersen, de Vreese, & Albaek, 2016;Coromina & Saris, 2009;Prior, 2013) or, more recently, on the accuracy and validity of self-reports compared to passive measures (LaCour & Vavreck, 2014;Prior, 2009; A previous version of this paper was presented at the 2016 annual meeting of the DGPuK methods division in Amsterdam, NL. I thank Marko Bachl for helpful comments and suggestions on the data analysis. Replication data and code are available at https://osf.io/t6z3qCorrespondence concerning this article should be addressed to Michael Scharkow, Department of Culture and Communication, Zeppelin University, Friedrichshafen, Germany. E-mail: michael.scharkow@zu.de RELIABILITY AND STABILITY OF MEDIA EXPOSURE 2 Scharkow, 2016;Wonneberger & Irazoqui, 2017). These studies often found low convergent validity and frequent misreporting. This paper follows a third strand of research which investigates the reliability and temporal stability of media exposure self-reports (Allen, 1981;Allen & Taylor, 1985;Lee, Hornik, & Hennessy, 2008). These two aspects are central to communication research for several reasons: (1) If self-reported media exposure is not reliable, the estimated media effects will likely be biased (Bartels, 1993;Scharkow & Bachl, 2017), and predictive or explanatory analyses of media exposure will be highly uncertain. (2) The temporal stability of media exposure is relevant both theoretically and empirically in order to understand media exposure and effects. If media exposure is exceedingly stable, there is little hope for experimental or sho...

Plurality in the Measurement of Social Media Use and Mental Health: An Exploratory Study Among Adolescents and Young Adults

“…& Reinecke, 2020; Sigerson & Cheng, 2018;Twenge & Farley, 2021), (2) how well different measures of SMU capture actual behavior and correlate with each other (Johannes et al, 2020;Scharkow, 2019), and (3) individual differences such as age and gender (Beyens et al, 2020;Twenge & Farley, 2021;Vannucci & Ohannessian, 2019). Depending on the measures used to tackle engagement with social media (e.g., screen time, number of followers or platforms), on specific platform features (followers on Instagram, or retweets on Twitter), and on developmental characteristics during early and later adolescence, a very different picture about the relation of SMU and other variables of interest can arise (Bij de Vaate et al, 2020;Dienlin & Johannes, 2020;Twenge & Farley, 2021).…”