ADAM: An Attentional Data Augmentation Method for Extreme Multi-label Text Classification

Zhang, Jiaxin; Liu, Jie; Chen, Shaowei; Lin, Shaoxin; Wang, Bingquan; Wang, Shanpeng

doi:10.1007/978-3-031-05933-9_11

Cited by 4 publications

(6 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, Xu and Le (2022) propose a feature generation method using a conditional variational autoencoder (VAE). Unfortunately, due to the label co-occurrence, it is challenging for these prior methods to handle MLTC (Wu et al 2020;Zhang et al 2022a). Instead of previous sample-level augmentation, we create a new pair-level augmentation strategy, which merely augments positive feature-label pairs for the tail-labels.…”

Section: Data Augmentationmentioning

confidence: 99%

“…One immediate approach to address the problem is data augmentation (DA) which can compensate the scarce data for tail-labels (Wang et al 2019;Chu et al 2020;Zhang et al 2020Zhang et al , 2022a. Meanwhile, DA has shown its effectiveness in many low-resource data scenarios, such as low-resource NLP (Wei and Zou 2019;Wu et al 2022), and zero/few-shot learning (Schwartz et al 2018;Keshari, Singh, and Vatsa 2020;Xu and Le 2022).…”

Section: Introductionmentioning

confidence: 99%

“…The Thirty-Seventh AAAI Conference on Artificial Intelligence (AAAI-23) However, because of the label co-occurrence, existing approaches of DA struggle in the multi-label scenario (Wu et al 2020;Zhang et al 2022a). A document usually contains several labels, making the selection for tail-labels no longer independent.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Label-Specific Feature Augmentation for Long-Tailed Multi-Label Text Classification

Xiao

Luo

et al. 2023

AAAI

View full text Add to dashboard Cite

Multi-label text classification (MLTC) involves tagging a document with its most relevant subset of labels from a label set. In real applications, labels usually follow a long-tailed distribution, where most labels (called as tail-label) only contain a small number of documents and limit the performance of MLTC. To facilitate this low-resource problem, researchers introduced a simple but effective strategy, data augmentation (DA). However, most existing DA approaches struggle in multi-label settings. The main reason is that the augmented documents for one label may inevitably influence the other co-occurring labels and further exaggerate the long-tailed problem. To mitigate this issue, we propose a new pair-level augmentation framework for MLTC, called Label-Specific Feature Augmentation (LSFA), which merely augments positive feature-label pairs for the tail-labels. LSFA contains two main parts. The first is for label-specific document representation learning in the high-level latent space, the second is for augmenting tail-label features in latent space by transferring the documents second-order statistics (intra-class semantic variations) from head labels to tail labels. At last, we design a new loss function for adjusting classifiers based on augmented datasets. The whole learning procedure can be effectively trained. Comprehensive experiments on benchmark datasets have shown that the proposed LSFA outperforms the state-of-the-art counterparts.

show abstract

Section: Data Augmentationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Label-Specific Feature Augmentation for Long-Tailed Multi-Label Text Classification

Xiao

Luo

et al. 2023

AAAI

View full text Add to dashboard Cite

show abstract

“…We first discuss the generation process, then we provide details on how the generative model is encouraged to generate high-quality data. Generation: Following prior works (Zhang et al, 2020), to generate synthetic data we employ GPT-2 (Radford et al, 2019) model. GPT-2 is a transformer-based language model pre-trained on 40 GB of textual data.…”

Section: Data Augmentationmentioning

confidence: 99%

“…One of the barriers to this task is the lack of labeled data. Inspired by the recent advances in the application of pre-trained language models to augment training data for low-resources tasks (Zhang et al, 2020;Yang et al, 2020;Peng et al, 2020;Kumar et al, 2020;Anaby-Tavor et al, 2020), we propose to employ the GPT-2 model to overcome the data scarcity of OSD. To address this limitation, we propose a novel model in which the OSD training data are augmented with the synthetic samples generated by a transformer-based language model.…”

Section: Introductionmentioning

confidence: 99%

Data Augmentation with Dual Training for Offensive Span Detection

Nouri¹

2022

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

Recognizing offensive text is an important requirement for every content management system, especially for social networks. While the majority of the prior work formulate this problem as text classification, i.e., if a text excerpt is offensive or not, in this work we propose a novel model for offensive span detection (OSD), whose goal is to identify the spans responsible for the offensive tone of the text. One of the challenges to train a model for this novel setting is the lack of enough training data. To address this limitation, in this work we propose a novel method in which the large-scale pretrained language model GPT-2 is employed to generate synthetic training data for OSD. In particular, we propose to train the GPT-2 model in a dual-training setting using the REINFORCE algorithm to generate in-domain, natural and diverse training samples. Extensive experiments on the benchmark dataset for OSD reveal the effectiveness of the proposed method.

show abstract

Taming Prompt-Based Data Augmentation for Long-Tailed Extreme Multi-Label Text Classification

Xu,

Song,

et al. 2024

ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

ADAM: An Attentional Data Augmentation Method for Extreme Multi-label Text Classification

Cited by 4 publications

References 8 publications

Label-Specific Feature Augmentation for Long-Tailed Multi-Label Text Classification

Label-Specific Feature Augmentation for Long-Tailed Multi-Label Text Classification

Data Augmentation with Dual Training for Offensive Span Detection

Taming Prompt-Based Data Augmentation for Long-Tailed Extreme Multi-Label Text Classification

Contact Info

Product

Resources

About