Multimodal Social Media Analysis for Gang Violence Prevention

Blandfort, Philipp; Patton, Desmond Upton; Frey, William R.; Karaman, Svebor; Bhargava, Surabhi; Lee, Fei-Tzin; Varia, Siddharth; Kedzie, Chris; Gaskell, Michael B.; Schifanella, Rossano; McKeown, Kathleen; Chang, Shih-Fu

doi:10.1609/icwsm.v13i01.3214

Cited by 10 publications

(4 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Content understanding. Finally, this work can aid further research in the broader field of social media content understanding, in the spirit of work such as Dewen et al (Dewan et al 2017) and Blandfort et al (Blandfort et al 2019).…”

Section: Related Workmentioning

confidence: 86%

Automatic Discovery of Political Meme Genres with Diverse Appearances

Theisen

Brogan

Thomas

et al. 2021

ICWSM

View full text Add to dashboard Cite

Forms of human communication are not static --- we expect some evolution in the way information is conveyed over time because of advances in technology. One example of this phenomenon is the image-based meme, which has emerged as a dominant form of political messaging in the past decade. While originally used to spread jokes on social media, memes are now having an outsized impact on public perception of world events. A significant challenge in automatic meme analysis has been the development of a strategy to match memes from within a single genre when the appearances of the images vary. Such variation is especially common in memes exhibiting mimicry. For example, when voters perform a common hand gesture to signal their support for a candidate. In this paper we introduce a scalable automated visual recognition pipeline for discovering political meme genres of diverse appearance. This pipeline can ingest meme images from a social network, apply computer vision-based techniques to extract local features and index new images into a database, and then organize the memes into related genres. To validate this approach, we perform a large case study on the 2019 Indonesian Presidential Election using a new dataset of over two million images collected from Twitter and Instagram. Results show that this approach can discover new meme genres with visually diverse images that share common stylistic elements, paving the way forward for further work in semantic analysis and content attribution.

show abstract

Section: Related Workmentioning

confidence: 86%

Automatic Discovery of Political Meme Genres with Diverse Appearances

Theisen

Brogan

Thomas

et al. 2021

ICWSM

View full text Add to dashboard Cite

show abstract

“…We utilized pseudonyms throughout this study when referring to specific participants. To further protect participant identity, we modified posts to make them unsearchable on the internet while preserving the meaning (Blandfort et al, 2019). This practice ensures that included posts cannot be used to search for and identify study participants.…”

Section: Discussionmentioning

confidence: 99%

On sex, drugs, and alcohol: a mixed-method analysis of youth posts on social media in the United States

Stevens

Bonett

Kenyatta³

et al. 2022

Journal of Children and Media

View full text Add to dashboard Cite

In the digital neighborhood, Black and Hispanic youth communicate about issues they face, including sex and substance use. This population of young people is also disproportionately burdened by negative consequences associated with sex and substance use behaviors even when their behavior is less risky. Given the increased likelihood of negative outcomes, we investigated how young people communicate about these behaviors in their digital social networks. This mixed methods study integrates a behavioral survey with a machine learning supported, qualitative content analysis of one year of Facebook and Twitter posts from 50 Black and Hispanic youth, with feedback from a youth advisory board. Participants who had previously had sex or drank alcohol were more likely to post about sex or alcohol, respectively. Moreover, compared to male youth , female youth posted significantly more about sex in general and about STIs, HIV, and pregnancy . Posts about sexual behavior often discussed trust in sexual partners and gendered views of sex. Posts linked alcohol and marijuana with stress relief and coping with grief, community violence, and distress. Understanding how youth think and communicate about sexual risk and substance use can inform the design of effective prevention efforts.

show abstract

“…However, user-generated information is usually multimodal and typically contains visual, audio, and textual information. Appropriate integration of multimodal features has also been shown in several works to improve the accuracy of data analysis [3,4] and it becomes a challenge to deal with the multimodal information in these messages [5].…”

Section: Introductionmentioning

confidence: 99%

AI-Empowered Multimodal Hierarchical Graph-Based Learning for Situation Awareness on Enhancing Disaster Responses

Chen,

Seng,

Ang

et al. 2024

Future Internet

View full text Add to dashboard Cite

Situational awareness (SA) is crucial in disaster response, enhancing the understanding of the environment. Social media, with its extensive user base, offers valuable real-time information for such scenarios. Although SA systems excel in extracting disaster-related details from user-generated content, a common limitation in prior approaches is their emphasis on single-modal extraction rather than embracing multi-modalities. This paper proposed a multimodal hierarchical graph-based situational awareness (MHGSA) system for comprehensive disaster event classification. Specifically, the proposed multimodal hierarchical graph contains nodes representing different disaster events and the features of the event nodes are extracted from the corresponding images and acoustic features. The proposed feature extraction modules with multi-branches for vision and audio features provide hierarchical node features for disaster events of different granularities, aiming to build a coarse-granularity classification task to constrain the model and enhance fine-granularity classification. The relationships between different disaster events in multi-modalities are learned by graph convolutional neural networks to enhance the system’s ability to recognize disaster events, thus enabling the system to fuse complex features of vision and audio. Experimental results illustrate the effectiveness of the proposed visual and audio feature extraction modules in single-modal scenarios. Furthermore, the MHGSA successfully fuses visual and audio features, yielding promising results in disaster event classification tasks.

show abstract

Multimodal Social Media Analysis for Gang Violence Prevention

Cited by 10 publications

References 21 publications

Automatic Discovery of Political Meme Genres with Diverse Appearances

Automatic Discovery of Political Meme Genres with Diverse Appearances

On sex, drugs, and alcohol: a mixed-method analysis of youth posts on social media in the United States

AI-Empowered Multimodal Hierarchical Graph-Based Learning for Situation Awareness on Enhancing Disaster Responses

Contact Info

Product

Resources

About