Enhancing Text Classification via Discovering Additional Semantic Clues from Logograms

Chen, Qian; Feng, Fuli; Wen, Lijie; Li, Lin; Chua, Tat-Seng

doi:10.1145/3397271.3401107

Cited by 3 publications

(8 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…TEXTCNN (Kim, 2014) is a classical classifier that uses convolutional neural networks (CNN) with scale-variant convolution filters to capture local textual features, which may potentially capture spurious correlations between certain keywords and categories. LECO (Qian et al, 2020b) utilizes the combination of the implicit encoding of deep linguistic information and the explicit encoding of morphological features, which would also capture the keyword bias inadvertently. Besides, it uses a sentence-level over-sampling mechanism (He and Garcia, 2009) to mitigate the label bias, and we further enhance it via a powerful word-level augmentation technique (EDA) (Wei and Zou, 2019) to mitigate the keyword bias, denoted as LECOEDA.…”

Section: Discussionmentioning

confidence: 99%

“…The core idea of CORSAIR is to train a "poisonous" text classifier regardless the dataset biases and post-adjust the biased predictions according to the causes of the biases in inference. It's worth mentioning that our proposed CORSAIR can be applied to almost any parameterized base model, including traditional one-stage classifiers (e.g., TEXTCNN (Kim, 2014), RCNN (Lai et al, 2015) and LECO (Qian et al, 2020b)) and currently prevalent two-stage classifiers 2 (e.g., ULM- Figure 1: The architecture of our proposed model-agnostic framework (CORSAIR). Specifically, CORSAIR first trains a base model on the training data directly so as to preserve the dataset biases in the trained model.…”

Section: Methodsmentioning

confidence: 99%

“…To solve the dataset bias issue, a straightforward solution is to perform data-level manipulations to prevent models from capturing the unintended dataset biases in model training, including data balance (Dixon et al, 2018;Geng et al, 2007;Chen et al, 2017;Sun et al, 2018;Rayhan et al, 2017;Nguyen et al, 2011) (a.k.a. resampling) and data augmentation (Wei and Zou, 2019;Qian et al, 2020b). Another common paradigm for text classification is typically to design model-level balancing mechanisms, including unbiased embedding (Bolukbasi et al, 2016;Kaneko and Bollegala, 2019), threshold correction (Kang et al, 2020;Provost, 2000;Calders and Verwer, 2010) and instance weighting Zhao et al, 2017;Jiang and Zhai, 2007).…”

Section: Related Workmentioning

confidence: 99%

“…To resolve the issues, an effective solution is to perform data-level manipulations (e.g., resampling (Qian et al, 2020b)), which effectively transforms a training set to a relatively balanced one before training. Another line of debiasing work typically designs model-level balancing mechanisms (e.g., reweighting ), aiming to adaptively decrease the influence of majority categories while increasing the minority during training.…”

Section: Introductionmentioning

confidence: 99%

“…Text classification, mapping text documents to a set of predefined categories, is a fundamental and important technique serving for many applications such as sentiment analysis (Qian et al, 2020b), partisanship recognition (Kiesel et al, 2019) and spam detection (Castillo et al, 2007). Machine learning models have become the default choice of solving text classification, owing to their ability to recognize the textual patterns from the labeled documents (Kim, 2014;Howard and Ruder, 2018).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Counterfactual Inference for Text Classification Debiasing

Chen¹,

Feng²,

Wen³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

Self Cite

View full text Add to dashboard Cite

Today's text classifiers inevitably suffer from unintended dataset biases, especially the document-level label bias and word-level keyword bias, which may hurt models' generalization. Many previous studies employed datalevel manipulations or model-level balancing mechanisms to recover unbiased distributions and thus prevent models from capturing the two types of biases. Unfortunately, they either suffer from the extra cost of data collection/selection/annotation or need an elaborate design of balancing strategies. Different from traditional factual inference in which debiasing occurs before or during training, counterfactual inference mitigates the influence brought by unintended confounders after training, which can make unbiased decisions with biased observations. Inspired by this, we propose a model-agnostic text classification debiasing framework -CORSAIR, which can effectively avoid employing data manipulations or designing balancing mechanisms. Concretely, CORSAIR first trains a base model on a training set directly, allowing the dataset biases "poison" the trained model. In inference, given a factual input document, COR-SAIR imagines its two counterfactual counterparts to distill and mitigate the two biases captured by the poisonous model. Extensive experiments demonstrate CORSAIR's effectiveness, generalizability and fairness. 1 * This work was partly done during Chen Qian's internship at Alibaba DAMO academy.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Counterfactual Inference for Text Classification Debiasing

Chen¹,

Feng²,

Wen³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

Self Cite

View full text Add to dashboard Cite

show abstract

A Training-Free Debiasing Framework with Counterfactual Reasoning for Conversational Emotion Detection

Tu,

Jing,

Liang

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Unintended dataset biases typically exist in existing Emotion Recognition in Conversations (ERC) datasets, including label bias, where models favor the majority class due to imbalanced training data, as well as the speaker and neutral word bias, where models make unfair predictions because of excessive correlations between specific neutral words or speakers and classes. However, previous studies in ERC generally focus on capturing context-sensitive and speaker-sensitive dependencies, ignoring the unintended dataset biases of data, which hampers the generalization and fairness in ERC. To address this issue, we propose a Training-Free Debiasing framework (TFD) that operates during prediction without additional training. To ensure compatibility with various ERC models, it does not balance data or modify the model structure. Instead, TFD extracts biases from the model by generating counterfactual utterances and contexts and mitigates them using simple yet empirically robust element-wise subtraction operations. Extensive experiments on three public datasets demonstrate that TFD effectively improves generalization ability and fairness across different ERC models 1 .

show abstract

Should We Rely on Entity Mentions for Relation Extraction? Debiasing Relation Extraction with Counterfactual Analysis

Wang¹,

Chen²,

Zhou³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent literature focuses on utilizing the entity information in the sentence-level relation extraction (RE), but this risks leaking superficial and spurious clues of relations. As a result, RE still suffers from unintended entity bias, i.e., the spurious correlation between entity mentions (names) and relations. Entity bias can mislead the RE models to extract the relations that do not exist in the text. To combat this issue, some previous work masks the entity mentions to prevent the RE models from overfitting entity mentions. However, this strategy degrades the RE performance because it loses the semantic information of entities. In this paper, we propose the CORE (Counterfactual Analysis based Relation Extraction) debiasing method that guides the RE models to focus on the main effects of textual context without losing the entity information. We first construct a causal graph for RE, which models the dependencies between variables in RE models. Then, we propose to conduct counterfactual analysis on our causal graph to distill and mitigate the entity bias, that captures the causal effects of specific entity mentions in each instance. Note that our CORE method is model-agnostic to debias existing RE systems during inference without changing their training processes. Extensive experimental results demonstrate that our CORE yields significant gains on both effectiveness and generalization for RE. The source code is provided at: https://github.com/vanoracai/CoRE.

show abstract

Enhancing Text Classification via Discovering Additional Semantic Clues from Logograms

Cited by 3 publications

References 18 publications

Counterfactual Inference for Text Classification Debiasing

Counterfactual Inference for Text Classification Debiasing

A Training-Free Debiasing Framework with Counterfactual Reasoning for Conversational Emotion Detection

Should We Rely on Entity Mentions for Relation Extraction? Debiasing Relation Extraction with Counterfactual Analysis

Contact Info

Product

Resources

About