Triaging content severity in online mental health forums

Abstract

Mental health forums are online communities where people express their issues and seek help from moderators and other users. In such forums, there are often posts with severe content indicating that the user is in acute distress and there is a risk of attempted self-harm. Moderators need to respond to these severe posts in a timely manner to prevent potential selfharm. However, the large volume of daily posted content makes it difficult for the moderators to locate and respond to these critical posts. We present a framework for triaging user content into four severity categories which are defined based on indications of self-harm ideation. Our models are based on a feature-rich classification framework which includes lexical, psycholinguistic, contextual and topic modeling features. Our approaches improve the state of the art in triaging the content severity in mental health forums by large margins (up to 17% improvement over the F-1 scores). Using the proposed model, we analyze the mental state of users and we show that overall, long-term users of the forum demonstrate a decreased severity of risk over time. Our analysis on the interaction of the moderators with the users further indicates that without an automatic way to identify critical content, it is indeed challenging for the moderators to provide timely response to the users in need.

Detecting Recovery Problems Just in Time: Application of Automated Linguistic Analysis and Supervised Machine Learning to an Online Substance Abuse Forum

“…Indeed, our results are more closely in line with studies that have used hand-labeled data for training. F-scores for our hybrid model are comparable with the best results achieved in a shared task challenge to flag messages for elevated suicide risk in a forum for Australian youth [ 50 ] and slightly lower than a follow-up study from the same forum that utilized an ensemble of feature extraction approaches (LIWC, topic modeling, meta-data, etc) [ 51 ]. However, it is important to note our more conservative approach of testing our model in a separate iteration of the forum with a separate patient population.…”

“…For instance, naturally occurring response patterns can be used, such as where Huh and colleagues [ 12 ] labeled as problematic those messages to which moderators had previously responded in a health support forum, using their linguistic features to classify new messages that moderators would likely be interested in. Alternately, human judgment can be used to generate each label in the training set, as was implemented in efforts to detect suicidality in an online discussion forum for youth [ 50 , 51 ]. This approach recognizes that moderators’ response patterns do not always clearly follow from the level risk a message indicates.…”

“…For instance, responding to a message need not represent concern, but could reflect interest in a particular topic or investment in an ongoing relationship. Using SVM, boosted decision trees, and other models, researchers were able to achieve F-scores over 0.9 in identifying messages urgently requiring response [ 50 , 51 ].…”

JASIST special issue on biomedical information retrieval

“…Cohan et al (), in a paper titled “Triaging Content Severity in Online Mental Health Forums,” the authors refer to the need to detect posts in mental health forums of users to prevent potential self‐harm. An approach for triaging user content into four severity categories that are defined based on an indication of self‐harm ideation is proposed.…”

User adoption of physician's replies in an online health community: An empirical study

“…Research on OHCs has focused on community theme analysis, sentiment analysis; physician's service charges; value creation from urban to rural areas; and so on (Cohan, Young, Yates, & Goharian, 2017;Goh, Gao, & Agarwal, 2016;Wu & Lu, 2018). Knowledge adoption in Q&A communities in general has attracted widespread academic attention, such as the information adoption of Wikipedia adoption (Shen, Cheung, & Lee, 2013), Baidu Knows (Jin, Yan, Li, & Li, 2016), et al However, there is little or no research on how users adopt satisfactory health information from physician's online replies.…”