Unpacking content moderation: The rise of social media platforms as online civil courts

Goanta, Catalina; Ortolani, Pietro

doi:10.4337/9781802203820.00017

Cited by 3 publications

(7 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since content moderation takes place in a complex ecosystem of policies, algorithmic recommender and demotion systems, automated detection tools and human enforcement (Bishop, 2019; Diaz and Hecht-Felella, 2021; Goanta and Ortolani, 2021), I frame flagging and de-platforming as moderation assemblages , or ‘the logics, processes and outcomes of social media content moderation’ (Gerrard and Thornham, 2020: 1279–1280), particularly when discussing how they come together to perpetrate online abuse and reinforce offline inequalities against marginalised groups. Used by Gerrard and Thornham (2020) to conceptualise how different elements in platform governance come together to perpetrate sexism, assemblages are the human and mechanical elements joining forces to generate specific results.…”

Section: The Assemblages Of Flagging and De-platformingmentioning

confidence: 99%

“…Flagging is an affordance ‘for reporting offensive content to a social media platform’ for it to be de-platformed (Crawford and Gillespie, 2016: 411). Similarly to affordances such as liking or commenting, it is a means social media and internet platforms offer users to take action (Graves, 2007) – in this case, to share feedback about the content Instagram and TikTok show their audiences (Goanta and Ortolani, 2021).…”

mentioning

confidence: 99%

“…On TikTok, too, reports seem to have influenced the censorship of specific accounts, particularly within the sex-positive, LGBTQIA+ and activist space (see Are, 2022; Perrett, 2021; Stokel-Walker, 2022), Similarly to how liking and commenting may determine the popularity of content and profiles (Glatt, 2022), flagging therefore has an element of power on Instagram and TikTok. It allows social networks to evade liability and to appear to be conferring power to audiences, potentially handing the already automated reins of platform governance to those who game the system to harass others (Are, 2022, 2023; Goanta and Ortolani, 2021). The misuse or malicious exploitation of flagging has therefore also been defined as organised flagging (Crawford and Gillespie, 2016), user-generated censorship (Peterson, 2013) and user-generated warfare (UGW) (Fiore-Silfvast, 2012).…”

mentioning

confidence: 99%

See 2 more Smart Citations

The assemblages of flagging and de-platforming against marginalised content creators

Are

2023

Convergence: The International Journal of Research into New Med

View full text Add to dashboard Cite

This study examines how de-platforming and flagging assemble to replicate offline inequalities, making content creators at the margins vulnerable to both online abuse and censorship on Instagram and TikTok. Highlighting gaps in online harms literature surrounding the misuse of this functionality, this paper frames misused or malicious flagging as online abuse through interviews with users who believed they were de-platformed this way, showcasing this practice’s emotional and financial impact on targets and creating a framework to identify it through users’ gossip.

show abstract

Section: The Assemblages Of Flagging and De-platformingmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

The assemblages of flagging and de-platforming against marginalised content creators

Are

2023

Convergence: The International Journal of Research into New Med

View full text Add to dashboard Cite

show abstract

“…As an additional issue, platforms typically deploy toxicity scoring in combination with opaque, one-way communication mechanisms [18,25,28,34]. For instance, a social network may remove content without providing information as to how it identified the content, which policies it violates, and what users can do to rectify the situation [5,35,44,47,66]. This can result in various undesired outcomes for users, including loss of revenue, influence, and attention, as well as total account suspension [12,55].…”

Section: Introductionmentioning

confidence: 99%

Recourse for Reclamation: Chatting with Generative Language Models

Chien,

Mckee,

Kay

et al. 2024

Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Researchers and developers increasingly rely on toxicity scoring to moderate generative language model outputs, in settings such as customer service, information retrieval, and content generation. However, toxicity scoring may render pertinent information inaccessible, rigidify or "value-lock" cultural norms, and prevent language reclamation processes, particularly for marginalized people. In this work, we extend the concept of algorithmic recourse to generative language models: we provide users a novel mechanism to achieve their desired prediction by dynamically setting thresholds for toxicity filtering. Users thereby exercise increased agency relative to interactions with the baseline system. A pilot study (𝑛 = 30) supports the potential of our proposed recourse mechanism, indicating improvements in usability compared to fixed-threshold toxicity-filtering of model outputs. Future work should explore the intersection of toxicity scoring, model controllability, user agency, and language reclamation processes-particularly with regard to the bias that many communities encounter when interacting with generative language models. CCS CONCEPTS• Human-centered computing → Human computer interaction (HCI); • Computing methodologies → Natural language processing.

show abstract

“…The consequences of exposure to such content are varied, with a large body of research associating exposure to negative online content with, for example, increased self-harm behaviours and suicidal ideation in vulnerable populations [ 5 ], elevation of extreme violence within communities [ 6 ], and widespread dissemination of fake news, terrorist propaganda and ensuing political polarisation across society [ 7 , 8 ]. Consequently, the regulation of online harm has risen high on the agenda of social media platforms and has been deemed to one of the greatest concerns for governments, supranational bodies, and international organisations [ 9 , 10 ].…”

Section: Introductionmentioning

confidence: 99%

Moral judgment of objectionable online content: Reporting decisions and punishment preferences on social media

Vahed,

Goanta,

Ortolani

et al. 2024

PLoS ONE

Self Cite

View full text Add to dashboard Cite

Harmful and inappropriate online content is prevalent, necessitating the need to understand how individuals judge and wish to mitigate the spread of negative content on social media. In an online study with a diverse sample of social media users (n = 294), we sought to elucidate factors that influence individuals’ evaluation of objectionable online content. Participants were presented with images varying in moral valence, each accompanied by an indicator of intention from an ostensible content poster. Half of the participants were assigned the role of user content moderator, while the remaining participants were instructed to respond as they normally would online. The study aimed to establish whether moral imagery, the intention of a content poster, and the perceived responsibility of social media users, affect judgments of objectionability, operationalized through both decisions to flag content and preferences to seek punishment of other users. Our findings reveal that moral imagery strongly influences users’ assessments of what is appropriate online content, with participants almost exclusively choosing to report and punish morally negative images. Poster intention also plays a significant role in user’s decisions, with greater objection shown to morally negative content when it has been shared by another user for the purpose of showing support for it. Bestowing a content moderation role affected reporting behaviour but not punishment preferences. We also explore individual user characteristics, finding a negative association between trust in social media platforms and reporting decisions. Conversely, a positive relationship was identified between trait empathy and reporting rates. Collectively, our insights highlight the complexity of social media users’ moderation decisions and preferences. The results advance understanding of moral judgments and punishment preferences online, and offer insights for platforms and regulatory bodies aiming to better understand social media users’ role in content moderation.

show abstract

Unpacking content moderation: The rise of social media platforms as online civil courts

Cited by 3 publications

References 0 publications

The assemblages of flagging and de-platforming against marginalised content creators

The assemblages of flagging and de-platforming against marginalised content creators

Recourse for Reclamation: Chatting with Generative Language Models

Moral judgment of objectionable online content: Reporting decisions and punishment preferences on social media

Contact Info

Product

Resources

About