SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles

Martino, Giovanni Da San; Barrón-Cedeño, Alberto; Wachsmuth, Henning; Petrov, Rostislav; Nakov, Preslav

doi:10.48550/arxiv.2009.02696

Cited by 5 publications

(3 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We analyse the results of two datasets in our approach, a collection of unlabelled articles from RT and the SemEval-2020 Task 11 dataset, curated by Martino et al [2], which is labelled according to the techniques they define in their other paper [1]. Due to the token restrictions imposed by the OpenAI API for the model we use, both datasets are filtered to ensure that no article exceeds 3000 characters.…”

Section: Methodsmentioning

confidence: 99%

Detecting Propaganda in News Articles Using Large Language Models

2024

Eng OA

View full text Add to dashboard Cite

The proliferation of media channels as a result of the information age has ushered in a new era of communication and access to information. However, this increased accessibility has also opened up new avenues for propaganda and the manipulation of public opinion. With the recent release of OpenAI's artificial intelligence chatbot, ChatGPT, users and the media are increasingly discovering and reporting on its range of novel capabilities. The most notable of these, such as answering technical questions, stem from its ability to perform advanced natural language processing and text generation. In this paper, we aim to assess the feasibility of using the underlying technology behind ChatGPT, Large Language Models (LLMs), to detect features of propaganda in news articles. The features we consider leverage the work of Martino et al., who define a list of 18 distinct propaganda techniques. For example, they outline the 'straw man' technique, which refers to the use of 'refuting an argument that was not presented' [1]. Based on these techniques, we develop a refined prompt that is coupled with news articles from Russia Today (RT), a prominent state-controlled news network, and from the labelled SemEval-2020 Task 11 dataset [2]. The prompt and article content are then sent to OpenAI’s gpt-3.5-turbo model to determine which propaganda techniques are present and to make a final judgement on whether the article is propaganda or not. We then qualitatively analyse a subset of the resulting output to determine whether LLMs can be used effectively in this way. With the results of the study, we aim to uncover whether such technologies show promise in detecting propaganda, and what sort of prompts lead to the most useful output. This has the potential to be useful for media consumers, for example, who could use our prompts to detect signs of propaganda in the articles they read.

show abstract

Section: Methodsmentioning

confidence: 99%

Detecting Propaganda in News Articles Using Large Language Models

2024

Eng OA

View full text Add to dashboard Cite

show abstract

“…Span Identification) -tikslas yra tekste atpažinti konkrečius propagandos fragmentus; antrojo uždavinio -Technikos Klasifikacijos (angl. Technique Classification) -tikslas -teksto fragmentui, kuris yra nustatytas kaip propagandinis, priskirti jame naudojamas technikas iš propagandos technikų sąrašo [3]. Šiame darbe išbandomi 3 pagrindiniai transformeriais pagrįsti, iš anksto apmokyti daugiakalbiai modeliai, apmokyti ir lietuvių kalba, sprendžiant paprastesnį, binarinį propagandos klasifikavimo uždavinį.…”

Section: įVadasunclassified

Propagandos atpažinimas lietuviškame tekste naudojant transformeriais pagrįstus, iš anksto apmokytus daugiakalbius modelius

Zaranka,

Korvel

2024

open-series

View full text Add to dashboard Cite

Didėjant informacijos kiekiui ir jos svarbai visuomenėje atsiranda vis didesnis poreikis automatinių įrankių, gebančių atpažinti propagandą. Dėl geopolitinės situacijos Lietuvos valstybė gali būti ypatingai pažeidžiama propagandinių mechanizmų, o automatinis jos atpažinimas lietuviškuose tekstuose yra nepakankamai ištyrinėta sritis. Šio darbo tikslas – išbandyti 3 pagrindinius transformeriais pagrįstus, iš anksto apmokytus daugiakalbius modelius propagandos atpažinimui. Sprendžiamas binarinis klasifikavimo uždavinys, priskiriant tekstui propagandinio arba nepropagandinio teksto klasę. LitLat, XLM-R ir mBERT modeliai adaptuoti apmokant ekspertų suanotuotu duomenų rinkiniu. Nors geriausią, 88,5 % F1 statistikos įvertį pavyko pasiekti adaptavus LitLat iš anksto apmokytą modelį, kiti šiame darbe adaptuoti modeliai pasiekia panašius rezultatus.

show abstract

“…The next iteration of the workshop, SemEval-2020 Task 11 [12], revised the FLC task by breaking it down into span identification and a revised 14-way propaganda technique classification. The top performing teams had an F1 of 51.55 for span identification and 62.07 for propaganda technique classification.…”

Section: Introductionmentioning

confidence: 99%

Uncovering deep-rooted cultural differences (UNCOVER)

Panasyuk,

Li,

Callison-Burch

2024

Disruptive Technologies in Information Sciences VIII

View full text Add to dashboard Cite

This study delves into the interconnected realms of Debates, Fake News, and Propaganda, with an emphasis on discerning prominent ideological underpinnings distinguishing Russian from English authors. Leveraging the advanced capabilities of Large Language Models (LLMs), particularly GPT-4, we process and analyze a large corpus of over 80,000 Wikipedia articles to unearth significant insights. Despite the inherent linguistic distinctions between Russian and English texts, our research highlights the adeptness of LLMs in bridging these variances. Our approach, includes translation, question generation and answering, along with emotional analysis, to probe the gathered information. A ranking metric based on the emotional content is used to assess the impact of our approach. Furthermore, our research identifies important limitations within existing data resources for propaganda identification. To address these challenges and foster future research, we present a curated synthetic dataset designed to encompass a diverse spectrum of topics and achieve balance across various propaganda types.

show abstract

SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles

Cited by 5 publications

References 53 publications

Detecting Propaganda in News Articles Using Large Language Models

Detecting Propaganda in News Articles Using Large Language Models

Propagandos atpažinimas lietuviškame tekste naudojant transformeriais pagrįstus, iš anksto apmokytus daugiakalbius modelius

Uncovering deep-rooted cultural differences (UNCOVER)

Contact Info

Product

Resources

About