Go Figure! Multi-task transformer-based architecture for metaphor detection using idioms: ETS team in 2020 metaphor shared task

Abstract

This paper describes the ETS entry to the 2020 Metaphor Detection shared task. Our contribution consists of a sequence of experiments using BERT, starting with a baseline, strengthening it by spell-correcting the TOEFL corpus, followed by a multi-task learning setting, where one of the tasks is the token-level metaphor classification as per the shared task, while the other is meant to provide additional training that we hypothesized to be relevant to the main task. In one case, out-of-domain data manually annotated for metaphor is used for the auxiliary task; in the other case, indomain data automatically annotated for idioms is used for the auxiliary task. Both multitask experiments yield promising results.

Perceptional and actional enrichment for metaphor detection with sensorimotor norms

“…More relevant works include Gao et al (2018) who employed Bi-LSTM as an encoder using GloVe (Pennington, Socher, and Manning 2014) and ELMo (Peters et al 2018) as text input representation; Brooks and Youssef (2020) built up an ensemble model of RNNs together with attention-based Bi-LSTMs for metaphor detection. Chen et al (2020) adopted BERT to obtain sentence embeddings, and then applied a linear layer with softmax to each token for metaphoricity predictions. DeepMet (Su et al 2020) utilized RoBERTa with various linguistic features.…”

Methods of Annotating and Identifying Metaphors in the Field of Natural Language Processing

“…Approaches with neural networks also use, in addition to neural networks, some of the word embeddings, being static [33][34][35][36][37][38][39][40] or contextual word embeddings [41][42][43][44][45][46][47][48][49][50][51] (Brief overview of word embeddings is presented in the section Neural networks and word embeddings). It is these models that have given the best results in identifying metaphors in recent years.…”

“…Approaching the metaphor identification as the sequence labeling task is most common in early research with the use of static word embeddings [33,[36][37][38], while there are also other approaches, e.g., enriching word embeddings with visual ones [34]. Recent research that utilizes contextual word embeddings are also experimenting with different approaches, e.g., reading comprehension task [41], learning from another type of figurative language [43], looking at the broader discourse [44], contrastive learning [45], or as relation extraction problem [51]. Lately, we notice that researchers are also trying to make the most of linguistic theories of metaphor (namely, MIP, Sectional Preferences and Conceptual Metaphor Theory) by implementing them in the architecture of their proposed models [40,[46][47][48][49][50] and it is these models that present the current state of the art in the field of metaphor identification.…”

“The Rodney Dangerfield of Stylistic Devices”: End-to-End Detection and Extraction of Vossian Antonomasia Using Neural Networks

“…Particularly, considering metaphor extraction as a sequence tagging task using neural network approaches has been studied recently. While Gao et al ( 2018 ), Dankers et al ( 2019 ); and Torres Rivera et al ( 2020 ) use contextualized word-embeddings and bi-directional LSTMs in their model architecture, Dankers et al ( 2019 ), Chen et al ( 2020 ), Gong et al ( 2020 ), and Liu et al ( 2020 ) make use of pre-trained contextual language models, for instance, BERT (Devlin et al, 2019 ), RoBERTa (Liu et al, 2019 ), or XLNet (Yang et al, 2019 ). Metonymy resolution has also found recent attention in the NLP community with the advent of pre-trained transformer models.…”