KOLD: Korean Offensive Language Dataset

Jeong, Younghoon; Oh, Juhyun; Lee, Jongwon; Ahn, Jaimeen; Moon, Jihyung; Park, Sungjoon; Oh, Alice

doi:10.18653/v1/2022.emnlp-main.744

Cited by 9 publications

(7 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared to the reported performance on other datasets, there was a broad range of values. For instance, the KOLD dataset [37] reported precision and recall rates of 50.8 and 47.8, respectively. In contrast, the best model in the "SemEval-2021 Task 5" toxic-spandetection competition, Ref.…”

Section: Discussionmentioning

confidence: 99%

“…They achieved a mean F1-score of 77.16% with PhoBERT large and 77.70% using XLM-RoBERTa large . Another recently released dataset is the Korean Offensive Language Dataset (KOLD) [37], which offers a hierarchical taxonomy and annotations at the span level for identifying toxic content. In addition to the taxonomy, similar to OLIF [15], Jeong et al [37] proposed the labeling of the target group, similar to the approach of HateXplain [29].…”

Section: Offensive and Toxic Spans' Datasetsmentioning

confidence: 99%

“…Another recently released dataset is the Korean Offensive Language Dataset (KOLD) [37], which offers a hierarchical taxonomy and annotations at the span level for identifying toxic content. In addition to the taxonomy, similar to OLIF [15], Jeong et al [37] proposed the labeling of the target group, similar to the approach of HateXplain [29]. Their approach to span detection involved fine-tuning the BERT base models using BIO-tags, enabling them to predict and identify the specific fragments of interest.…”

Section: Offensive and Toxic Spans' Datasetsmentioning

confidence: 99%

See 2 more Smart Citations

Offensive Text Span Detection in Romanian Comments Using Large Language Models

Paraschiv,

Ion,

Dascalu

2023

Information

View full text Add to dashboard Cite

The advent of online platforms and services has revolutionized communication, enabling users to share opinions and ideas seamlessly. However, this convenience has also brought about a surge in offensive and harmful language across various communication mediums. In response, social platforms have turned to automated methods to identify offensive content. A critical research question emerges when investigating the role of specific text spans within comments in conveying offensive characteristics. This paper conducted a comprehensive investigation into detecting offensive text spans in Romanian language comments using Transformer encoders and Large Language Models (LLMs). We introduced an extensive dataset of 4800 Romanian comments annotated with offensive text spans. Moreover, we explored the impact of varying model sizes, architectures, and training data volumes on the performance of offensive text span detection, providing valuable insights for determining the optimal configuration. The results argue for the effectiveness of BERT pre-trained models for this span-detection task, showcasing their superior performance. We further investigated the impact of different sample-retrieval strategies for few-shot learning using LLMs based on vector text representations. The analysis highlights important insights and trade-offs in leveraging LLMs for offensive-language-detection tasks.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Offensive and Toxic Spans' Datasetsmentioning

confidence: 99%

Section: Offensive and Toxic Spans' Datasetsmentioning

confidence: 99%

See 1 more Smart Citation

Offensive Text Span Detection in Romanian Comments Using Large Language Models

Paraschiv,

Ion,

Dascalu

2023

Information

View full text Add to dashboard Cite

show abstract

“…Many studies have presented benchmark models and provide a number of moral frameworks for reducing risk prior to product release. For example, studies aim to prevent physical harm (Levy et al, 2022) or hate speech by establishing benchmarks (Jeong et al, 2022), consistently contributing to practical design references. However, the paper, so-called "the Salmon paper" criticizes some of existing benchmark datasets that are designed to measure stereotyping (Su et al, 2021).…”

Section: Related Workmentioning

confidence: 99%

Evaluating and Improving Value Judgments in AI: A Scenario-Based Study on Large Language Models’ Depiction of Social Conventions

You,

Suh

2024

ICWSM

View full text Add to dashboard Cite

The adoption of generative AI technologies is swiftly expanding. Services employing both linguistic and multimodal models are evolving, offering users increasingly precise responses. Consequently, human reliance on these technologies is expected to grow rapidly. With the premise that people will be impacted by the output of AI, we explored approaches to help AI output produce better results. Initially, we evaluated how contemporary AI services competitively meet user needs, then examined society's depiction as mirrored by Large Language Models (LLMs). We did a query experiment, querying about social conventions in various countries and eliciting a one-word response. We compared the LLMs' value judgments with public data and suggested a model of decision-making in value-conflicting scenarios which could be adopted for future machine value judgments. This paper advocates for a practical approach to using AI as a tool for investigating other remote worlds. This research has significance in implicitly rejecting the notion of AI making value judgments and instead arguing a more critical perspective on the environment that defers judgemental capabilities to individuals. We anticipate this study will empower anyone, regardless of their capacity, to receive safe and accurate value judgment-based outputs effectively.

show abstract

“…Offensive language can vary greatly depending on cultural backgrounds. While most multilingual OLD datasets are constructed by filtering a predefined list of offensive words (Zampieri et al, 2019;Sigurbergsson and Derczynski, 2020;Jeong et al, 2022;Deng et al, 2022), certain offensive words are culturally specific. For example, OLD models trained on American cultural contexts may struggle to effectively detect offensive words like "m*adarchod" and "pr*sstitute" in Indian texts (Ghosh et al, 2021;Santy et al, 2023).…”

Section: Introductionmentioning

confidence: 99%

Cultural Compass: Predicting Transfer Learning Success in Offensive Language Detection with Cultural Features

Zhou,

Karamolegkou,

Chen

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

The increasing ubiquity of language technology necessitates a shift towards considering cultural diversity in the machine learning realm, particularly for subjective tasks that rely heavily on cultural nuances, such as Offensive Language Detection (OLD). Current understanding underscores that these tasks are substantially influenced by cultural values, however, a notable gap exists in determining if cultural features can accurately predict the success of cross-cultural transfer learning for such subjective tasks. Addressing this, our study delves into the intersection of cultural features and transfer learning effectiveness. The findings reveal that cultural value surveys indeed possess a predictive power for cross-cultural transfer learning success in OLD tasks and that it can be further improved using offensive word distance. Based on these results, we advocate for the integration of cultural information into datasets. Additionally, we recommend leveraging data sources rich in cultural information, such as surveys, to enhance cultural adaptability. Our research signifies a step forward in the quest for more inclusive, culturally sensitive language technologies. 1Warning: This paper discusses examples of offensive content. The authors do not support the use of offensive language, nor any of the offensive representations quoted below.

show abstract

KOLD: Korean Offensive Language Dataset

Cited by 9 publications

References 0 publications

Offensive Text Span Detection in Romanian Comments Using Large Language Models

Offensive Text Span Detection in Romanian Comments Using Large Language Models

Evaluating and Improving Value Judgments in AI: A Scenario-Based Study on Large Language Models’ Depiction of Social Conventions

Cultural Compass: Predicting Transfer Learning Success in Offensive Language Detection with Cultural Features

Contact Info

Product

Resources

About