Modeling Semantic Similarity between Metaphor Terms of Visual vs. Linguistic Metaphors through Flickr Tag Distributions

Abstract

This study aims at modeling the semantic similarity between metaphor terms by means of a distributional method based on a Big Data stream: Flickr tags. As explained in the article, this distributional model, Flickr Distributional Tagspace (FDT), captures primarily relational similarity between concept pairs, that is, between tags that appear in similar tagsets (and therefore in similar pictures). A long established view in metaphor theory claims that metaphors pertain to the conceptual dimension of meaning, but while different models aim at explaining how language constructs and represents metaphorical conceptual structures, we still know very little about how other modalities (for example, images) achieve metaphor construction and expression. A comprehensive theory, which argues in favor of the conceptual nature of metaphor, cannot afford to be biased toward the analysis and modeling of one specific modality of expression, thus neglecting potential modalityspecific differences. The present study, conducted through FDT, found that visual and linguistic metaphors behave differently, in that the similarity between two aligned concepts in a visual metaphor appears to be significantly higher than the similarity between two concepts aligned in a linguistic metaphor (which, in turn, does not differ substantially from the similarity between two randomly paired concepts). These findings suggest that the relational similarity between two metaphor terms (captured and modeled through FDT) is crucial for visual metaphors but not for linguistic metaphors. An additional content analysis, also reported here, shows that the type of semantic information encoded in the related tags (i.e., the contexts on which the contingency matrices of this distributional method are built) differs, in relation to the modality of the metaphor: while situationrelated and entity-related features are typically associated with concepts aligned in visual metaphors, introspections, and taxonomic features are typically associated with concepts aligned in linguistic metaphors. INTRODUCTIONMetaphor is recognized nowadays as a cognitive mechanism, rather than a simple figure of speech (Lakoff and Johnson, 1980;Lakoff, 1993). Through metaphors, we understand one concept in terms of another concept, the latter being generally perceived as easier or more concrete. On this view, metaphoric expressions, such as our love is at a crossroads, we can't turn back now, together we are not going anywhere, suggest that when we deal with the abstract concept of LOVE (by convention, capital letters are used when referring to a concept or a conceptual domain), we might somehow envision a concrete journey going along a path. This conceptualization allows us to extend the metaphor, and imagine the two lovers as two travelers, the difficulties in the relationship as impediments on the journey, and the lovers' goals as the travel destination. From this complex and rather subconscious conceptual structure (LOVE-IS-JOURNEY), we would then derive metaphoric linguistic expressi...

The N1–N2–LPC Pattern in Processing Advertising Pictorial Metaphors: An ERP Study

“…To study the neurocognitive mechanism of pictorial metaphors, different methods have been attempted, such as eye tracking (Indurkhya and Ojha, 2013), EEG (Ma et al, 2016), fMRI (Ojha et al, 2017) and distributional method of semantics (Bolognesi, 2016). Comparatively, ERP is advantageous in terms of high time-resolution and its application to explore the temporal course and dynamic mechanism of pictorial metaphor (Kutas and Hillyard, 1980; Hillyard and Kutas, 1983), which is of great significance to the study of advertising visual stimuli, as the typical observation time for print advertising is two seconds in an editorial context (Pieters et al, 1996).…”

The Flickr frequency norms: What 17 years of images tagged online tell us about lexical processing

“…Notably, the use of Flickr in our field of research is not new. For example, Bolognesi (2014), by using a distributional approach, built up a semantic space based on Flickr tag co-occurrences (i.e., Flickr Distributional Space, see also Bolognesi, 2016Bolognesi, , 2017 and showed that an inherently visual domain -such as the domain of colour -is better mapped by the Flickr Distributional Space rather than distributional spaces based on solely textual information (Baroni & Lenci, 2010;Landauer & Dumais, 1997). Moreover, Flickr is a gold mine in computer vision research: with its huge numbers of tagged images, it serves as an excellent resource for training convolutional neural networks (e.g., T. Chen et al, 2014;X.…”

A test of indirect grounding of abstract concepts using multimodal distributional semantics

“…Furthermore, another type of approach has been proposed which utilizes visual information without directly computing visual vectors from images. Bolognesi's (2016Bolognesi's ( , 2017) Flickr R Distributional Tagspace uses the co-occurrence statistics between user-generated tags that appear in the same image to generate word vectors. In this paper, we use multimodal distributional semantics to model the hybrid view of conceptual representation.…”