How should we investigate variation in the relation between social media and well-being?

Abstract

The study of the relation between social media use and well-being is at a critical junction. Many researchers find small to no associations, yet policymakers and public stakeholders keep asking for more evidence. One way the field is reacting is by inspecting the variation around average relations – with the goal of describing individual social media users. Here, we argue that such an approach risks losing sight of the most important outcomes of a quantitative social science: estimates of the average relation in a large group. Our analysis begins by describing how the field got to this point. Then, we explain the problems of the current approach of studying variation. Next, we propose a principled approach to quantify, interpret, and explain variation in average relations: (1) conducting model comparisons, (2) defining a region of practical equivalence and testing the theoretical distribution of relations against that region, (3) defining a smallest effect size of interest and comparing it against the theoretical distribution. We close with recommendations to either study moderators as systematic factors that explain variation or to conduct N = 1 studies and qualitative research.

Three objections to a novel paradigm in social media effects research

“…For one, the group interprets percentages that are not uniform as evidence that associations are not the same for everyone and we therefore cannot ignore heterogeneity. As observed previously (Johannes et al, 2021): Variation is the norm, not the exception, and we do not immediately know what to do with these findings because no preexisting theory of social media influence proposed that effects would be identical to everyone. Instead of acknowledging that the heterogeneity in observed associations might reflect differ-ential confounding of the actual causal effect, theoretically important moderators, and residual between-person differences resulting from researchers' lack of complete knowledge of the participants, the authors assert that the field is on the cusp of a personalized "media effects paradigm" (Valkenburg et al, 2021a, p. 74, emphasis ours) akin to those theorized for medicine and education.…”

“…With all of this in mind, we underscore the valuable and collegial exchanges we have had with the authors whose work we are here critiquing (Johannes et al, 2021;. The stakes for understanding well-being in the digital age are simply too high for personal acrimony or ideological fiefdoms.…”

“…As we have argued (Johannes et al, 2021), one way to resolve this tension is to use information about heterogeneity in considering meaningful moderators of social media associations (e.g. Bolger et al, 2019).…”

“…(Valkenburg, 2022, p. 5) Because aggregate statistics purportedly cannot adequately describe individuals, this group hopes that others will follow them in what they alternatively call a "personalized media effects paradigm" (Valkenburg et al, 2021a, p. 74) or a "causal effect heterogeneity paradigm" (Valkenburg, 2022, p. 5). We previously outlined concerns with this body of work, including that it conflates studying heterogeneity with idiographic research (Johannes et al, 2021) (see for the authors' response). Here, we focus on a pervasive methodological issue in that body of work that we did not previously address.…”

Eight Hypotheses on Technology Use and Psychosocial Wellbeing: A Bicultural Phenomenological Study of Gaming during the COVID-19 Pandemic

“…That said, it remains unknown how those social functions were affected by the pandemic. Although multiplayer gaming with social features clearly increased in total quantity during times of social restrictions (Vuorre et al, 2021 ), it is not known if and how the nature of social interaction in videogames changed, or how the effect heterogeneity (of various wellbeing-related constructs) manifest in individual cases (Johannes et al, 2021b ). Different changes in social dynamics can be expected to occur in people’s gaming routines during lockdown (see Domahidi & Quandt, 2015 ).…”