Representation of Linguistic Form and Function in Recurrent Neural Networks

Abstract

We present novel methods for analyzing the activation patterns of RNNs from a linguistic point of view and explore the types of linguistic structure they learn. As a case study, we use a multi-task gated recurrent network architecture consisting of two parallel pathways with shared word embeddings trained on predicting the representations of the visual scene corresponding to an input sentence, and predicting the next word in the same sentence. Based on our proposed method to estimate the amount of contribution of individual tokens in the input to the final prediction of the networks we show that the image prediction pathway: a) is sensitive to the information structure of the sentence b) pays selective attention to lexical categories and grammatical functions that carry semantic information c) learns to treat the same input token differently depending on its grammatical functions in the sentence. In contrast the language model is comparatively more sensitive to words with a syntactic function. Furthermore, we propose methods to explore the function of individual hidden units in RNNs and show that the two pathways of the architecture in our case study contain specialized units tuned to patterns informative for the task, some of which can carry activations to later time steps to encode long-term dependencies.

What do Entity-Centric Models Learn? Insights from Entity Linking in Multi-Party Dialogue

“…Analysis of Neural Network Models. Our work joins a recent strand in NLP that systematically analyzes what different neural network models learn about language (Linzen et al, 2016;Kádár et al, 2017;Conneau et al, 2018;Gulordava et al, 2018b;Nematzadeh et al, 2018, a.o.). This work, like ours, has yielded both positive and negative results: There is evidence that they learn complex linguistic phenomena of morphological and syntactic nature, like long distance agreement (Gulordava et al, 2018b;Giulianelli et al, 2018), but less evidence that they learn how language relates to situations; for instance, Nematzadeh et al (2018) show that memory-augmented neural models fail on tasks that require keeping track of inconsistent states of the world.…”

Interrogating the Explanatory Power of Attention in Neural Machine Translation

“…Another method to measure relevance is by removing the input, and tracking the difference in in network's output (Li et al, 2016b). While these methods focus on explaining a model's decision, Shi et al (2016); Kádár et al (2017); Calvillo and Crocker (2018) investigate how a particular concept is represented in the network. Analyzing and interpreting the attention mechanism in NLP (Koehn and Knowles, 2017;Ghader and Monz, 2017;Tang and Nivre, 2018;Clark et al, 2019;Vig and Belinkov, 2019) is another direction that has drawn major interest.…”

Models of Visually Grounded Speech Signal Pay Attention to Nouns: A Bilingual Experiment on English and Japanese

“…This paper focuses on computational models of visually grounded speech that were introduced by [14,4]. Learned representations of such models were analyzed by [11,7,4]: [11] introduced novel methods for interpreting the activation patterns of recurrent neural networks (RNN) in a model of visually grounded meaning representation from textual and visual input and showed that RNN pay attention to word tokens belonging to specific lexical categories. [4] found that final layers tend to encode semantic information whereas lower layers tend to encode form-related information.…”