Bigrams and BiLSTMs Two Neural Networks for Sequential Metaphor Detection

Abstract

We present and compare two alternative deep neural architectures to perform word-level metaphor detection on text: a bi-LSTM model and a new structure based on recursive feedforward concatenation of the input. We discuss different versions of such models and the effect that input manipulation -specifically, reducing the length of sentences and introducing concreteness scores for words -have on their performance.

Neural Metaphor Detection in Context

“…Many neural models with various features and architectures were introduced in the 2018 VUA Metaphor Detection Shared Task. They include LSTM-based models and CRFs augmented by linguistic features, such as WordNet, POS tags, concreteness score, unigrams, lemmas, verb clusters, and sentence-length manipulation (Swarnkar and Singh, 2018;Pramanick et al, 2018;Mosolova et al, 2018;Bizzoni and Ghanimifard, 2018;Wu et al, 2018). Researchers also studied different word embeddings, such as embeddings trained from corpora representing different levels of language mastery (Stemle and Onysko, 2018) and binarized vectors that reflect the General Inquirer dictionary category of a word (Mykowiecka et al, 2018).…”

End-to-End Sequential Metaphor Identification Inspired by Linguistic Theories

“…In terms of computational metaphor identification, feature-engineering has been widely discussed (Leong et al, 2018). Unigrams, imageability, concreteness, abstractness, word embedding and semantic classes are features, commonly employed by supervised machine learning (Turney et al, 2011;Assaf et al, 2013;Tsvetkov et al, 2014;Klebanov et al, 2016), deep learning (Rei et al, 2017;Gutierrez et al, 2017;Bizzoni and Ghanimifard, 2018) and unsupervised leaning Mao et al, 2018) approaches.…”

A Report on the 2018 VUA Metaphor Detection Shared Task

“…OCOTA (Bizzoni and Ghanimifard, 2018) experimented with a deep neural network composed of a Bi-LSTM preceded and followed by fully connected layers, as well as a simpler model that has a sequence of fully connected neural networks. The authors also experiment with word embeddings trained on various data, with explicit features based on concreteness, and with preprocessing that addresses variability in sentence length.…”