Mixed Methods for Human–Computer Interactions Research: An Iterative Study Using Reddit and Social Media

Abstract

Researchers have called for additional empirical studies associated with video games. However, every game is novel in terms of mechanics, content, context, and agency; known variables may be operationalized differently based on the game involved. It is incumbent upon researchers to leverage or create the best tools when extracting data from games. This article models instrument development of a scale intended to catalog users’ actions in a novel context (i.e., the game League of Legends) using a mixed methods approach. Specifically, this work outlines data collection and validation strategies using an online social news aggregation, rating, and discussion resource (i.e., Reddit), involving multiple cycles to elicit expert input and queries to generate consensus from expert gamers, followed by analysis of responses and an exploratory factor analysis for scale construction. Reddit provided three unique functions relative to the process: (a) access to experts who informed the development of scale items, (b) a social space and mechanism to validate scale items, and (c) the opportunity to capture data necessary to establish the psychometric properties of the instrument. Findings associated with scale development (i.e., item generation, theoretical and psychometric analyses) are presented. Overall, implications for instrument development in continually evolving contexts are discussed.

Adjusting sails for changing winds: exploring Reddit use for professional purposes in higher education

“…The social media platform Reddit is highly popular (Schrader et al, 2020 ), ranking as the sixth most visited website in the United States and 14th globally (Alexa Internet, 2022 ). Reddit hosts more than 50 million daily active users, with more than 13 billion posts and comments (Reddit Inc., 2022).…”

“…Reddit allows asynchronous threaded conversations; therefore, contributors can generate posts and comments (i.e., responses to others’ posts) in a threaded format. For users who prefer not to post or comment, Reddit provides a voting feature where users can either upvote or downvote a post or comment (Carpenter et al, 2018 ; Schrader et al, 2020 ; Staudt Willet & Carpenter, 2020 ).…”

“…All other members are welcome to post or comment, but the moderator may delete any inappropriate posts or comments. Those who do not wish to post or comment can participate in a less demanding way by upvoting or downvoting subreddit content (Carpenter et al, 2018 ; Schrader et al, 2020 ; Staudt Willet & Carpenter, 2020 ). In addition, the anonymity feature of Reddit provides a safe environment where people can freely interact with the topic, content, and others without feeling afraid to be judged (Carpenter & Staudt Willet, 2021 ).…”