A Human-Centered Systematic Literature Review of the Computational Approaches for Online Sexual Risk Detection

Abstract

In the era of big data and artificial intelligence, online risk detection has become a popular research topic. From detecting online harassment to the sexual predation of youth, the state-of-the-art in computational risk detection has the potential to protect particularly vulnerable populations from online victimization. Yet, this is a high-risk, high-reward endeavor that requires a systematic and human-centered approach to synthesize disparate bodies of research across different application domains, so that we can identify best practices, potential gaps, and set a strategic research agenda for leveraging these approaches in a way that betters society. Therefore, we conducted a comprehensive literature review to analyze 73 peer-reviewed articles on computational approaches utilizing text or meta-data/multimedia for online sexual risk detection. We identified sexual grooming (75%), sex trafficking (12%), and sexual harassment and/or abuse (12%) as the three types of sexual risk detection present in the extant literature. Furthermore, we found that the majority (93%) of this work has focused on identifying sexual predators after-the-fact, rather than taking more nuanced approaches to identify potential victims and problematic patterns that could be used to prevent victimization before it occurs. Many studies rely on public datasets (82%) and third-party annotators (33%) to establish ground truth and train their algorithms. Finally, the majority of this work (78%) mostly focused on algorithmic performance evaluation of their model and rarely (4%) evaluate these systems with real users. Thus, we urge computational risk detection researchers to integrate more human-centered approaches to both developing and evaluating sexual risk detection algorithms to ensure the broader societal impacts of this important work.

Social Media Communication and Network Correlates of HIV Infection and Transmission Risks Among Black Sexual Minority Men: Cross-sectional Digital Epidemiology Study

“…Digital epidemiology rests on the idea that digital communications such as social media posts are made in organic environments and that what people post about reflects activities, behaviors, and dispositions that are relevant to various health outcomes. Demonstration studies have put these assumptions to the test and showed that what people talk about on social media platforms can be used to predict health outcomes such as depression [28], sexual risk [29], and HIV and STI hotspots [30].…”

Human-Centered Artificial Intelligence, a review

“…Finally, the vast majority of the results of this research (78%) relied on algorithmic performance evaluations of their models, with just 4% evaluating these systems with real customers. They recommended computational risk detection researchers to utilize more humancentered techniques when creating and testing sexual risk detection algorithms to guarantee that their essential work has larger social repercussions [36].…”

Ethical Considerations When Constructing Participatory Design Protocols for Social Robots

“…The field of human-computer interaction (HCI) has studied and proposed various technology-facilitated tools for sexual wellness as well as solutions to sexual violence. To the latter, examples include wearable devices that emit bad odors to deter rapists [5], panic buttons on mobile devices to alert authorities [1], AI to detect sexual predators who attempt to coerce minors into meeting face-to-face [6] and dating apps transformed into sexual consent apps [4]. However, these solutions are typically either known to be unsuccessful, severely limited, or are lacking empirical evidence of their impact.…”