Predicting Age Groups of Reddit Users Based on Posting Behavior and Metadata: Classification Model Development and Validation

Abstract

Background Social media are important for monitoring perceptions of public health issues and for educating target audiences about health; however, limited information about the demographics of social media users makes it challenging to identify conversations among target audiences and limits how well social media can be used for public health surveillance and education outreach efforts. Certain social media platforms provide demographic information on followers of a user account, if given, but they are not always disclosed, and researchers have developed machine learning algorithms to predict social media users’ demographic characteristics, mainly for Twitter. To date, there has been limited research on predicting the demographic characteristics of Reddit users. Objective We aimed to develop a machine learning algorithm that predicts the age segment of Reddit users, as either adolescents or adults, based on publicly available data. Methods This study was conducted between January and September 2020 using publicly available Reddit posts as input data. We manually labeled Reddit users’ age by identifying and reviewing public posts in which Reddit users self-reported their age. We then collected sample posts, comments, and metadata for the labeled user accounts and created variables to capture linguistic patterns, posting behavior, and account details that would distinguish the adolescent age group (aged 13 to 20 years) from the adult age group (aged 21 to 54 years). We split the data into training (n=1660) and test sets (n=415) and performed 5-fold cross validation on the training set to select hyperparameters and perform feature selection. We ran multiple classification algorithms and tested the performance of the models (precision, recall, F1 score) in predicting the age segments of the users in the labeled data. To evaluate associations between each feature and the outcome, we calculated means and confidence intervals and compared the two age groups, with 2-sample t tests, for each transformed model feature. Results The gradient boosted trees classifier performed the best, with an F1 score of 0.78. The test set precision and recall scores were 0.79 and 0.89, respectively, for the adolescent group (n=254) and 0.78 and 0.63, respectively, for the adult group (n=161). The most important feature in the model was the number of sentences per comment (permutation score: mean 0.100, SD 0.004). Members of the adolescent age group tended to have created accounts more recently, have higher proportions of submissions and comments in the r/teenagers subreddit, and post more in subreddits with higher subscriber counts than those in the adult group. Conclusions We created a Reddit age prediction algorithm with competitive accuracy using publicly available data, suggesting machine learning methods can help public health agencies identify age-related target audiences on Reddit. Our results also suggest that there are characteristics of Reddit users’ posting behavior, linguistic patterns, and account features that distinguish adolescents from adults.

An Investigation of Age-Differentiated Conversations About Electronic Nicotine Delivery Systems on Reddit

“…A previously developed age prediction model was used to predict the age group for each author as either UA (13–20 years), OLA (21–54 years), or uncertain. 11 These categories were used to examine the differences in conversations depending on whether the user was OLA to use tobacco. The lower bound was selected because Reddit users must be aged ≥13 years, and those aged >54 years could not be appropriately classified because of the small number of individuals who fell into this category during the development of the model.…”

“…A full list of the variables used in the model, as well as further background on other variables considered, variable importance, and model performance, can be found in Chew et al. 11 Because the model does not produce perfect predictions (test set F1 score, ∼0.79), we reduced the likelihood that the model returned false positives by only considering predictions with a predicted probability >0.6 for either age group. This process of rejecting predictions for which the model is most uncertain is referred to as classification with a reject option 13 in the literature.…”

“… 10 To better understand conversations of tobacco public education target audiences on Reddit, Chew et al. 11 developed an algorithm that examines users’ posts and metadata to predict and categorize Reddit users’ ages into 1 of 2 groups: 13–20 years (i.e., underage [UA]) or 21–54 years (i.e., of legal age [OLA]). These 2 age groups were used to separate users’ legal use of tobacco products and to provide an appropriate model because there were very few age references for those aged >54 years.…”

“…These 2 age groups were used to separate users’ legal use of tobacco products and to provide an appropriate model because there were very few age references for those aged >54 years. This exploratory study, using the Chew and colleagues 11 algorithm, investigates ENDS conversations, with a focus on flavor restriction and Tobacco 21 policy discussions for posts originating from predicted UA and OLA groups.…”

Emotional and cognitive changes surrounding online depression identity claims

“…A byproduct of this process is an increasing number of identity claims, in which a person shares a specific facet of their identity with a new audience. Social media sites are scattered with identity claims, as people seek to provide information about their identity as context; in lieu of traditional reporting mechanisms like surveys, identity claims found using pattern matching have been used to label users with demographic information such as gender and age [1][2][3] and mental health status [4][5][6].…”