Cross-Lingual Event Detection via Optimized Adversarial Training

Guzman-Nateras, Luis; Nguyen, Minh Van; Nguyen, Thien

doi:10.18653/v1/2022.naacl-main.409

Cited by 7 publications

(9 citation statements)

References 20 publications

(33 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Base Model: Following prior work (Guzman-Nateras et al, 2022), we formulate ED as a sequence labeling problem to facilitate cross-lingual transfer learning (CLTL). Given an input sentence of n tokens W = {w 1 , w 2 , .…”

Section: Modelmentioning

confidence: 99%

“…We utilize the XLM-R base model with 768 dimensions in the hidden vectors to be comparable with previous work Guzman-Nateras et al, 2022). We tune the hyperparameters for our model over the development data using the EN→ZH language pair.…”

Section: Modelmentioning

confidence: 99%

“…The key challenging in CLTL for ED is to bridge the gap between representation spaces for the source and target languages to enable knowledge transfer. As such, previous work in this area has focused on the representation matching approaches, aiming to transform the representations for the source and target languages into a common space to train language-general ED models over source language data, e.g., via similarity regularization or adversarial learning (Guzman-Nateras et al, 2022). However, to achieve a common space, the source-language representations might need to sacrifice languagespecific information, which can involve discriminative features for event prediction due to the mix of information in the representations for input text.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Transitioning Representations between Languages for Cross-lingual Event Detection via Langevin Dynamics

Nguyen,

Dernoncourt

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

Cross-lingual transfer learning (CLTL) for event detection (ED) aims to develop models in high-resource source languages that can be directly applied to produce effective performance for lower-resource target languages. Previous research in this area has focused on representation matching methods to develop a language-universal representation space into which source-and target-language example representations can be mapped to achieve crosslingual transfer. However, as this approach modifies the representations for the sourcelanguage examples, the models might lose discriminative features for ED that are learned over training data of the source language to prevent effective predictions. To this end, our work introduces a novel approach for cross-lingual ED where we only aim to transition the representations for the target-language examples into the source-language space, thus preserving the representations in the source language and their discriminative information. Our method introduces Langevin Dynamics to perform representation transition and a semantic preservation framework to retain event type features during the transition process. Extensive experiments over three languages demonstrate the state-of-the-art performance for ED in CLTL.

show abstract

Section: Modelmentioning

confidence: 99%

Section: Modelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Transitioning Representations between Languages for Cross-lingual Event Detection via Langevin Dynamics

Nguyen,

Dernoncourt

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

show abstract

“…Unlike such prior work, we introduce a new word-label alignment perspective using OT for ED. Finally, some recent work has utilized OT for character/word/example alignment problems (Dou and Neubig, 2021;Xu et al, 2021;Veyseh et al, 2021aVeyseh et al, , 2022Guzman-Nateras et al, 2022). However, none of them explores OT for word-label alignment in ED.…”

Section: Related Workmentioning

confidence: 99%

Word-Label Alignment for Event Detection: A New Perspective via Optimal Transport

Veyseh¹,

Nguyen²

2022

Proceedings of the 11th Joint Conference on Lexical and Computational Semantics

Self Cite

View full text Add to dashboard Cite

Event Detection (ED) aims to identify mentions/triggers of real world events in text. In the literature, this task is modeled as a sequencelabeling or word-prediction problem. In this work, we present a novel formulation in which ED is modeled as a word-label alignment task. In particular, given the words in a sentence and possible event types, the objective is to infer an alignment matrix in which event trigger words are aligned with the most likely event types. Moreover, we show that this new perspective facilitates the incorporation of wordlabel alignment biases to improve alignment matrix for ED. Novel alignment biases and Optimal Transport are introduced to solve our alignment problem for ED. We conduct experiments on a benchmark dataset to demonstrate the effectiveness of the proposed model for ED.

show abstract

“…Unlike such prior work, we introduce a new word-label alignment perspective using OT for ED. Finally, some recent work has utilized OT for character/word/example alignment problems (Dou and Neubig, 2021;Veyseh et al, 2021aVeyseh et al, , 2022Guzman-Nateras et al, 2022). However, none of them explores OT for word-label alignment in ED.…”

Section: Related Workmentioning

confidence: 99%

Proceedings of the 11th Joint Conference on Lexical and Computational Semantics

2022

View full text Add to dashboard Cite

In recent years, NLP has made what appears to be incredible progress, with performance even surpassing human performance on some benchmarks. How should we interpret these advances? Have these models achieved language "understanding"? Operating on the premise that "understanding" will necessarily involve the capacity to extract and deploy meaning information, in this talk I will discuss a series of projects leveraging targeted tests to examine NLP models' ability to capture meaning in a systematic fashion. I will first discuss work probing model representations for compositional meaning, with a particular focus on disentangling compositional information from encoding of lexical properties. I'll then explore models' ability to extract and use meaning information when executing the basic pretraining task of word prediction in context. In all cases, these investigations apply tests that prioritize control of unwanted cues, so as to target the desired model capabilities with greater precision. The results of these studies suggest that although models show a good deal of sensitivity to word-level information, and to certain semantic and syntactic distinctions, when subjected to controlled tests they show little sign of representing higher-level compositional meaning, or of being able to retain and deploy such information robustly during word prediction. Instead, models show signs of heuristic predictive strategies that are unsurprising given their training, but that differ critically from systematic understanding of meaning. I will discuss potential implications of these findings with respect to the goals of achieving "understanding" with currently dominant pre-training paradigms.Bio: Allyson Ettinger is an Assistant Professor in the Department of Linguistics at the University of Chicago. Her interdisciplinary work combines methods and insights from cognitive science, linguistics, and computer science to examine meaning extraction and predictive processes executed during language processing in artificial intelligence systems and in humans. She received her PhD in Linguistics from the University of Maryland, and spent a year as research faculty at the Toyota Technological Institute at Chicago (TTIC) before beginning her appointment at the University of Chicago. She holds an additional courtesy appointment at TTIC.

show abstract

Cross-Lingual Event Detection via Optimized Adversarial Training

Cited by 7 publications

References 20 publications

Transitioning Representations between Languages for Cross-lingual Event Detection via Langevin Dynamics

Transitioning Representations between Languages for Cross-lingual Event Detection via Langevin Dynamics

Word-Label Alignment for Event Detection: A New Perspective via Optimal Transport

Proceedings of the 11th Joint Conference on Lexical and Computational Semantics

Contact Info

Product

Resources

About