Improving Stance Detection with Multi-Dataset Learning and Knowledge Distillation

Abstract

Stance detection determines whether the author of a text is in favor of, against or neutral to a specific target and provides valuable insights into important events such as legalization of abortion. Despite significant progress on this task, one of the remaining challenges is the scarcity of annotations. Besides, most previous works focused on a hardlabel training in which meaningful similarities among categories are discarded during training. To address these challenges, first, we evaluate a multi-target and a multi-dataset training settings by training one model on each dataset and datasets of different domains, respectively. We show that models can learn more universal representations with respect to targets in these settings. Second, we investigate the knowledge distillation in stance detection and observe that transferring knowledge from a teacher model to a student model can be beneficial in our proposed training settings. Moreover, we propose an Adaptive Knowledge Distillation (AKD) method that applies instance-specific temperature scaling to the teacher and student predictions. Results show that the multi-dataset model performs best on all datasets and it can be further improved by the proposed AKD, outperforming the state-of-the-art by a large margin. We publicly release our code. 1

Implicit Stance Detection with Hashtag Semantic Enrichment

“…The low accuracy of clustering risks propagating errors, and the reliance on abundant unlabeled data makes unsupervised methods less adaptable to cross-target and zero-shot settings. Li et al [32] proposed incorporating LLM to generate the explanation of hashtags and enhance model performance. However, directly utilizing the knowledge from large models is restricted to their training corpus and may propagate errors.…”

“…The result is shown in Table 5. Following previous work [6,32,41], we select a specific target as the test set, with the remaining task data as the training set. For example, we use →DT to denote DT as the test set, with the remaining targets (JB, SEM16-h and COV-h) as the training data.…”

“…Ts-CoT [40] first proposes CoT methods with LLMs for stance detection. • KASD [32] proposes to augment the hashtag by utilizing LLMs.…”

Infusing Knowledge from Wikipedia to Enhance Stance Detection

“…People have wide-ranging background knowledge regarding various targets and use it to infer the implicit stance in a statement. However, machines by default do not have such knowledge and previous works on stance detection (Allaway and McKeown, 2020;Allaway et al, 2021;Liang et al, 2021;Augenstein et al, 2016;Siddiqua et al, 2019;Sun et al, 2018;Li et al, 2021b;Hardalov et al, 2021) fail to incorporate such knowledge in modeling stances.…”

Infusing Knowledge from Wikipedia to Enhance Stance Detection

“…People have wide-ranging background knowledge regarding various targets and use it to infer the implicit stance in a statement. However, machines by default do not have such knowledge and previous works on stance detection (Allaway and McKeown, 2020;Allaway et al, 2021;Liang et al, 2021;Augenstein et al, 2016;Siddiqua et al, 2019;Sun et al, 2018;Li et al, 2021b;Hardalov et al, 2021) fail to incorporate such knowledge in modeling stances.…”