AdvCodeMix: Adversarial Attack on Code-Mixed Data

Abstract

Research on adversarial attacks are becoming widely popular in the recent years. One of the unexplored areas where prior research is lacking is the effect of adversarial attacks on code-mixed data. Therefore, in the present work, we have explained the first generalized framework on text perturbation to attack code-mixed classification models in a black-box setting. We rely on various perturbation techniques that preserve the semantic structures of the sentences and also obscure the attacks from the perception of a human user. The present methodology leverages the importance of a token to decide where to attack by employing various perturbation strategies. We test our strategies on various sentiment classification models trained on Bengali-English and Hindi-English code-mixed datasets, and reduce their F1-scores by nearly 51% and 53% respectively, which can be further reduced if a larger number of tokens are perturbed in a given sentence.

Elevating Code-mixed Text Handling through Auditory Information of Words

“…Increasing phenomena of code-mixing on social media platforms have also motivated researchers to analyze the adversarial robustness of code-mixed models. Authors in (Das et al, 2022) exposed the vulnerability of code-mixed classifiers by performing an adversarial attack based on subword perturbations, character repetition, and word language change. However, there is no attempt to enhance the adversarial robustness of code-mixed text classifiers against these perturbations.…”

“…, w n , with ground truth label y, and a target model M (S) = y, the goal of the adversary is to perform an untargeted attack, i.e., find an adversarial sample S adv , causing M to perform misclassification, i.e., M (S)! = y. Adversaries attack the model using phonetic perturbations in line with the prior work Das et al (2022). Design goals: Based on the aforementioned adversary model, our proposed framework (SMLM and SAMLM) must meet the robustness and accuracy requirements.…”

“…Managing such variations within text data are typically achieved through pre-processing techniques, such as data augmentation and normalization (Kusampudi et al, 2021), which necessitates the utilization of human-annotated dictionaries and can entail a significant investment in manual annotation efforts. It has been observed that traditional techniques for processing and analysis of code-mixed content may prove ineffective in cases where the spelling of a word varies from those present in the corpus or dictionary (Das et al, 2022).…”

“…Although transformer based pre-trained models (Devlin et al, 2018;Liu et al, 2019b) have proven to be largely effective for most of the tasks in Natural Language Processing (NLP) , it has been shown that even such models are not robust enough to handle small perturbations in spelling (Das et al, 2022). Such perturbations have been used to perform adversarial attacks on even the transformers based language models.…”

“…We then fine-tune our pretrained models on downstream classification tasks based on code-mixed Hinglish (Hindi+English) and Benglish (Bengali+English) datasets. We perform phonetic perturbation-based attacks following Das et al (2022) and find that our SMLM and SAMLM pre-trained models are more robust to such adversarial attacks. We observe a lower drop in performance in both models when compared to the base BERT and RoBERTa models after the attack.…”