Stimulus Parameters Underlying Sound‐Symbolic Mapping of Auditory Pseudowords to Visual Shapes

Abstract

Sound symbolism refers to non‐arbitrary mappings between the sounds of words and their meanings and is often studied by pairing auditory pseudowords such as “maluma” and “takete” with rounded and pointed visual shapes, respectively. However, it is unclear what auditory properties of pseudowords contribute to their perception as rounded or pointed. Here, we compared perceptual ratings of the roundedness/pointedness of large sets of pseudowords and shapes to their acoustic and visual properties using a novel application of representational similarity analysis (RSA). Representational dissimilarity matrices (RDMs) of the auditory and visual ratings of roundedness/pointedness were significantly correlated crossmodally. The auditory perceptual RDM correlated significantly with RDMs of spectral tilt, the temporal fast Fourier transform (FFT), and the speech envelope. Conventional correlational analyses showed that ratings of pseudowords transitioned from rounded to pointed as vocal roughness (as measured by the harmonics‐to‐noise ratio, pulse number, fraction of unvoiced frames, mean autocorrelation, shimmer, and jitter) increased. The visual perceptual RDM correlated significantly with RDMs of global indices of visual shape (the simple matching coefficient, image silhouette, image outlines, and Jaccard distance). Crossmodally, the RDMs of the auditory spectral parameters correlated weakly but significantly with those of the global indices of visual shape. Our work establishes the utility of RSA for analysis of large stimulus sets and offers novel insights into the stimulus parameters underlying sound symbolism, showing that sound‐to‐shape mapping is driven by acoustic properties of pseudowords and suggesting audiovisual cross‐modal correspondence as a basis for language users' sensitivity to this type of sound symbolism.

Phonation Types Matter in Sound Symbolism

“…The current study used descriptive categories for phonation types. Employing a set of specific acoustic measures will make a more fine‐grained observation possible (see Lacey et al., 2020; Perlman & Lupyan, 2018).…”

“…The majority of sound symbolism studies focus on segments, and the sound symbolism of suprasegmentals, including phonation types, has not been extensively investigated (for rare exceptions, see Dingemanse, Schuerman, Reinisch, Tufvesson, & Mitterer, 2016; Lacey et al., 2020; Nygaard, Herold, & Namy, 2009; Perlman, Dale, & Lupyan, 2015). However, the non‐arbitrary uses of some phonation types have been repeatedly noted in the phonetic and sociolinguistic literature, not necessarily in relation to sound symbolism (see Esling, Moisik, Benner, & Crevier‐Buchman, 2019; Laver, 1994).…”

Neural Basis of the Sound-Symbolic Crossmodal Correspondence Between Auditory Pseudowords and Visual Shapes

“…Thus, in soundsymbolic language, magnitude representations may serve to link the sound structure of spoken language with object shapes (which can vary along dimensions of pointedness, roundedness, or other shape attributes). That judgments of the sounds of language vary systematically along such dimensions is supported by findings that pseudowords with varying collections of phonetic features are readily placed along continua of pointedness and roundedness (McCormick et al, 2015), and that ratings transition from rounded to pointed with increasing vocal roughness, as indexed by a number of acoustic parameters (Lacey et al, 2020). On this account, activity related to sound symbolism might be expected in the intraparietal sulcus (IPS), a region involved in processing both numerical and non-numerical (e.g., luminance) magnitude (Sathian et al, 1999;Eger et al, 2003;Walsh, 2003;Pinel et al, 2004;Piazza et al, 2004Piazza et al, , 2007Sokolowski et al, 2017).…”

“…We used the most rounded visual shape, rated 1.6, but had to use the 11 th most pointed visual shape, rated 6.4, in order to match the number of protuberances (four) between the shapes. A more detailed analysis of the pseudowords and shapes can be found in Lacey et al, (2020) and the stimuli themselves are available at https://osf.io/ekpgh/. While the pseudowords differed in duration by 67ms, approximately 12%, sound segments in language naturally differ in duration and digitally altering them to match in duration can cause them to sound artificially rapid or prolonged.…”

Vowel Length Expands Perceptual and Emotional Evaluations in Written Japanese Sound-Symbolic Words

“…Previous studies of the bouba-kiki effect showed that some pseudowords (e.g., bouba ) were linked with round figures, whereas other pseudowords (e.g., kiki ) were linked with spiky figures [ 6 , 7 ]. One study reported that auditory features of spoken pseudowords cross-modal associated with visual features of figures [ 8 ]. Linguistic features (specific consonants and vowels) can increase the expected gustatory features [ 9 ].…”