Stable Learning via Differentiated Variable Decorrelation

Shen, Zuo‐Jun Max; Cui, Peng; Liu, Jiashuo; Zhang, Tong; Li, Bo; Chen, Zhitang

doi:10.1145/3394486.3403269

“…However, these methods are built on simple regressions or regular neural networks such as CNNs, but GNNs have more complex architectures and properties needed to be considered. We also notice that [6] propose a differentiated variable decorrelation term for linear regression. However, this decorrelation term requires multiple environments with different correlations between stable variables and unstable variables available in the training stage while our method does not require this prior knowledge.…”

Section: Related Workmentioning

confidence: 93%

“…Based on the causal view analysis of the decorrelation regularizer, we theoretically prove that the weights of variables can be differentiated by the regression coefficients. Compared with existing decorrelation methods [5], [6], the proposed regularizer is able to remove the spurious correlation while maintaining a higher effective sample size and requiring less prior knowledge. Moreover, to better combine the decorrelation regularizer with existing GNN architecture, the theoretical result shows that adding the regularizer to the embeddings learned by the penultimate layer could be both theoretically sound and flexible.…”

Section: Introductionmentioning

confidence: 99%

Debiased Graph Neural Networks with Agnostic Label Selection Bias

Fan¹,

Wang²,

Shi³

et al. 2022

Preprint

0

View full text Add to dashboard Cite

“…Causal Inference: Causal inference (Pearl et al, 2016;Rubin, 2019) has been applied in many areas, including visual tasks (Tang et al, 2020b;Abbasnejad et al, 2020;Niu et al, 2021;Zhang et al, 2020a;Yue et al, 2020;Nan et al, 2021b), model robustness and stable learning (Srivastava et al, 2020;Zhang et al, 2020a;Shen et al, 2020;Yu et al, 2020;Dong et al, 2020), generation , language understanding (Feng et al, 2021b), and recommendation systems (Jesson et al, 2020;Zhang et al, 2021d;Wang et al, 2021b;Ding et al, 2021). Works most related to ours are (Zeng et al, 2020;Wang and Culotta, 2021) that generates counterfactuals for weakly-supervised NER and text classifications, respectively.…”

Section: Related Workmentioning

confidence: 99%

Uncovering Main Causalities for Long-tailed Information Extraction

Nan¹,

Zeng²,

Qiao³

et al. 2021

Preprint

View full text Add to dashboard Cite

Information Extraction (IE) aims to extract structural information from unstructured texts. In practice, long-tailed distributions caused by the selection bias of a dataset, may lead to incorrect correlations, also known as spurious correlations, between entities and labels in the conventional likelihood models. This motivates us to propose counterfactual IE (CFIE), a novel framework that aims to uncover the main causalities behind data in the view of causal inference. Specifically, 1) we first introduce a unified structural causal model (SCM) for various IE tasks, describing the relationships among variables; 2) with our SCM, we then generate counterfactuals based on an explicit language structure to better calculate the direct causal effect during the inference stage; 3) we further propose a novel debiasing approach to yield more robust predictions. Experiments on three IE tasks across five public datasets show the effectiveness of our CFIE model in mitigating the spurious correlation issues. * * Contributed equally. Accepted as a long paper in the main conference of EMNLP 2021 (Conference on Empirical Methods in Natural Language Processing).

show abstract

“…Causal Inference: Causal inference (Pearl et al, 2016;Rubin, 2019) has been applied in many areas, including visual tasks (Tang et al, 2020b;Abbasnejad et al, 2020;Niu et al, 2021;Zhang et al, 2020a;Yue et al, 2020;Nan et al, 2021b), model robustness and stable learning (Srivastava et al, 2020;Zhang et al, 2020a;Shen et al, 2020;Yu et al, 2020;Dong et al, 2020), generation , language understanding (Feng et al, 2021b), and recommendation systems (Jesson et al, 2020;Zhang et al, 2021d;Wang et al, 2021b;Ding et al, 2021). Works most related to ours are (Zeng et al, 2020;Wang and Culotta, 2021) that generates counterfactuals for weakly-supervised NER and text classifications, respectively.…”

Section: Related Workmentioning

confidence: 99%

Uncovering Main Causalities for Long-tailed Information Extraction

Nan¹,

Zeng²,

Qiao³

et al. 2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Information Extraction (IE) aims to extract structural information from unstructured texts. In practice, long-tailed distributions caused by the selection bias of a dataset, may lead to incorrect correlations, also known as spurious correlations, between entities and labels in the conventional likelihood models. This motivates us to propose counterfactual IE (CFIE), a novel framework that aims to uncover the main causalities behind data in the view of causal inference. Specifically, 1) we first introduce a unified structural causal model (SCM) for various IE tasks, describing the relationships among variables; 2) with our SCM, we then generate counterfactuals based on an explicit language structure to better calculate the direct causal effect during the inference stage; 3) we further propose a novel debiasing approach to yield more robust predictions. Experiments on three IE tasks across five public datasets show the effectiveness of our CFIE model in mitigating the spurious correlation issues. * * Contributed equally.

show abstract

Stable Learning via Differentiated Variable Decorrelation

Cited by 34 publications

References 19 publications

Debiased Graph Neural Networks with Agnostic Label Selection Bias

Debiased Graph Neural Networks with Agnostic Label Selection Bias

Uncovering Main Causalities for Long-tailed Information Extraction

Uncovering Main Causalities for Long-tailed Information Extraction

Contact Info

Product

Resources

About