Towards Open Domain Event Trigger Identification using Adversarial Domain Adaptation

Abstract

We tackle the task of building supervised event trigger identification models which can generalize better across domains. Our work leverages the adversarial domain adaptation (ADA) framework to introduce domain-invariance. ADA uses adversarial training to construct representations that are predictive for trigger identification, but not predictive of the example's domain. It requires no labeled data from the target domain, making it completely unsupervised. Experiments with two domains (English literature and news) show that ADA leads to an average F1 score improvement of 3.9 on outof-domain data. Our best performing model (BERT-A) reaches 44-49 F1 across both domains, using no labeled target data. Preliminary experiments reveal that finetuning on 1% labeled data, followed by self-training leads to substantial improvement, reaching 51.5 and 67.2 F1 on literature and news respectively. 1

Adapting Event Extractors to Medical Data: Bridging the Covariate Shift

“…The MLP layer is 100-dimensional. These values are consistent with the setup in Naik and Rosé (2020). BERT-ADA: The domain predictor (adversary) is a 3-layer MLP with each layer having a dimensionality of 100 and ReLU activations between layers.…”

“…We propose a new method (LIW) which relies on instance weighting via language model likelihood, and contrast it with adversarial domain adaptation (ADA) and domain adaptive fine-tuning (DAFT). These two techniques have shown promise on sequence labeling tasks (Gui et al, 2017;Han and Eisenstein, 2019;Naik and Rosé, 2020), and offer an interesting contrast between approaches that jointly perform alignment and task training (ADA) and approaches that perform these steps sequentially (DAFT). Comparing all three techniques also provides us the opportunity to study which methods adapt better to different kinds of shifts between source and target domains (e.g., shifts in vocabulary, syntax, etc.…”

Neural Unsupervised Domain Adaptation in NLP—A Survey

“…DANNs have been applied in many NLP tasks in the last few years, mainly to sentiment classification (e.g., Ganin et al (2016), Li et al (2018a), Shen et al (2018), Rocha andLopes Cardoso (2019), Ghoshal et al (2020), to name a few), but recently to many other tasks as well: language identification (Li et al, 2018a), natural language inference (Rocha and Lopes Cardoso, 2019), POS tagging (Yasunaga et al, 2018), parsing (Sato et al, 2017), trigger identification (Naik and Rose, 2020), relation extraction Fu et al, 2017;Rios et al, 2018), and other (binary) text classification tasks like relevancy identification (Alam et al, 2018a), machine reading comprehension , stance detection (Xu et al, 2019), and duplicate question detection (Shah et al, 2018). This makes DANNs the most widely used UDA approach in NLP, as illustrated in Table 1.…”

Learning Cross-lingual Representations for Event Coreference Resolution with Multi-view Alignment and Optimal Transport

“…Compared to such prior work, this paper presents two novel approaches to improve the language generalization of representation vectors based on multi-view alignment and OT. Finally, our work involves LANN that bears some similarity with DANN models in domain adaptation research of machine learning (Ganin et al, 2016;Bousmalis et al, 2016;Fu et al, 2017;Naik and Rose, 2020;. Compared to such work, our work explores a new dimension of adversarial networks for language-invariant representation learning for texts in ECR.…”