No abstract

The concept of sound iconicity implies that phonemes are intrinsically associated with non-acoustic phenomena, such as emotional expression, object size or shape, or other perceptual features. In this respect, sound iconicity is related to other forms of cross-modal associations in which stimuli from different sensory modalities are associated with each other due to the implicitly perceived correspondence of their primal features. One prominent example is the association between vowels, categorized according to their place of articulation, and size, with back vowels being associated with bigness and front vowels with smallness. However, to date the relative influence of perceptual and conceptual cognitive processing on this association is not clear. To bridge this gap, three experiments were conducted in which associations between nonsense words and pictures of animals or emotional body postures were tested. In these experiments participants had to infer the relation between visual stimuli and the notion of size from the content of the pictures, while directly perceivable features did not support–or even contradicted–the predicted association. Results show that implicit associations between articulatory-acoustic characteristics of phonemes and pictures are mainly influenced by semantic features, i.e., the content of a picture, whereas the influence of perceivable features, i.e., size or shape, is overridden. This suggests that abstract semantic concepts can function as an interface between different sensory modalities, facilitating cross-modal associations.

We tested the influence of perceptual features on semantic associations between the acoustic characteristics of vowels and the notion of size. To this end, we designed an experiment in which we manipulated size on two dissociable levels: the physical size of the pictures presented during the experiment (perceptual level) and the implied size of the objects depicted in the pictures (semantic level). Participants performed an Implicit Association Test in which the pictures of small objects were larger than those of large objects – that is, the actual size ratio on the semantic level was inverted on the perceptual level. Our results suggest that participants matched visual and acoustic stimuli in accordance with the content of the pictures (i.e., the inferred size of the depicted object), whereas directly perceivable features (i.e., the physical size of the picture) had only a marginal influence on participants’ performance. Moreover, as the experiment has been conducted at two different sites (Japan and Germany), the results also suggest that the participants’ cultural background or mother tongue had only a negligible influence on the effect. Our results, therefore, support the assumption that associations across sensory modalities can be motivated by the semantic interpretation of presemantic stimuli.

Research on the relation between sound and meaning in language has reported substantial evidence for implicit associations between articulatory-acoustic characteristics of phonemes and emotions. In the present study, we specifically tested the relation between the acoustic properties of a text and its emotional tone as perceived by readers. To this end, we asked participants to assess the emotional tone of single stanzas extracted from a large variety of poems. The selected stanzas had either an extremely high, a neutral, or an extremely low average formant dispersion. To assess the average formant dispersion per stanza, all words were phonetically transcribed and the distance between the first and second formant per vowel was calculated. Building on a long tradition of research on associations between sound frequency on the one hand and non-acoustic concepts such as size, strength, or happiness on the other hand, we hypothesized that stanzas with an extremely high average formant dispersion would be rated lower on items referring to Potency (dominance) and higher on items referring to Activity (arousal) and Evaluation (emotional valence). The results confirmed our hypotheses for the dimensions of Potency and Evaluation, but not for the dimension of Activity. We conclude that, at least in poetic language, extreme values of acoustic features of vowels are a significant predictor for the emotional tone of a text.

No abstract