Emotional pictures and sounds: a review of multimodal interactions of emotion cues in multiple domains

Gerdes, Antje B. M.; Wieser, Matthias J.; Alpers, Georg W.

doi:10.3389/fpsyg.2014.01351

Cited by 91 publications

(77 citation statements)

References 120 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…As such, it is perhaps not so surprising to find that the emotional mediation account should prove to have so much explanatory validity for complex auditory stimuli, at least when compared to its role in explaining the correspondences that have been documented between the presumably much less emotionally-valenced simple visual and auditory stimuli used in much of the research in this area (though see [68][69][70][71]). Whatever the explanation, the emotional mediation account of crossmodal correspondences/influences has undoubtedly become an increasingly common theme (or explanatory approach) in the literature in this area in recent years (e.g., see [72][73][74]).…”

Section: Correspondences Involving Complex Auditorymentioning

confidence: 99%

Simple and complex crossmodal correspondences involving audition

Spence

2020

Acoust. Sci. & Tech.

View full text Add to dashboard Cite

The last few years have seen an explosion of interest from researchers in the crossmodal correspondences, defined as the surprising connections that the majority of people share between seemingly-unrelated stimuli presented in different sensory modalities. Intriguingly, many of the crossmodal correspondences that have been documented/studied to date have involved audition as one of the corresponding modalities. In fact, auditory pitch may well be the single most commonly studied dimension in correspondences research thus far. That said, relatively separate literatures have focused on the crossmodal correspondences involving simple versus more complex auditory stimuli. In this review, I summarize the evidence in this area and consider the relative explanatory power of the various different accounts (statistical, structural, semantic, and emotional) that have been put forward to explain the correspondences. The suggestion is made that the relative contributions of the different accounts likely differs in the case of correspondences involving simple versus more complex stimuli (i.e., pure tones vs. short musical excerpts). Furthermore, the consequences of presenting corresponding versus non-corresponding stimuli likely also differ in the two cases. In particular, while crossmodal correspondences may facilitate binding (i.e., multisensory integration) in the case of simple stimuli, the combination of more complex stimuli (such as, for example, musical excerpts and paintings) may instead be processed more fluently when the component stimuli correspond. Finally, attention is drawn to the fact that the existence of a crossmodal correspondence does not in-and-ofitself necessarily imply that a crossmodal influence of one modality on the perception of stimuli in the other will also be observed.

show abstract

Section: Correspondences Involving Complex Auditorymentioning

confidence: 99%

Simple and complex crossmodal correspondences involving audition

Spence

2020

Acoust. Sci. & Tech.

View full text Add to dashboard Cite

show abstract

“…Although we live in highly diffuse and "vivid" multisensory environments, and despite the growing interest from different application domains, most studies on human emotional responses to environmental characteristics still focus on a number of well-defined and restricted sensory aspects of the environment (typically under highly controlled conditions). As a result, we still lack systematic knowledge about successful multisensory interventions that elicit desirable outcomes (Barrett, Barrett, & Davies, 2013;Gerdes, Wieser, & Alpers, 2014;Jain & Bagdare, 2011;Oakes & North, 2008;Spence, Puccinelli, Grewal, & Roggeveen, 2014;Turley & Milliman, 2000). Environmental characteristics such as luminosity of light sources, the nature and level of ambient noise and acoustics, the presence of specific odors, color hues and shades, and materials and atmospheric factors such as temperature and humidity, all generate sensory input, and combined contribute to specific reactions in the observer (Biggers & Pryer, 1982;Franz, 2006).…”

Section: Introductionmentioning

confidence: 99%

Emotional Responses to Multisensory Environmental Stimuli

Schreuder¹,

Erp²,

Toet³

et al. 2016

SAGE Open

126

View full text Add to dashboard Cite

How we perceive our environment affects the way we feel and behave. The impressions of our ambient environment are influenced by its entire spectrum of physical characteristics (e.g., luminosity, sound, scents, temperature) in a dynamic and interactive way. The ability to manipulate the sensory aspects of an environment such that people feel comfortable or exhibit a desired behavior is gaining interest and social relevance. Although much is known about the sensory effects of individual environmental characteristics, their combined effects are not a priori evident due to a wide range of non-linear interactions in the processing of sensory cues. As a result, it is currently not known how different environmental characteristics should be combined to effectively induce desired emotional and behavioral effects. To gain more insight into this matter, we performed a literature review on the emotional effects of multisensory stimulation. Although we found some interesting mechanisms, the outcome also reveals that empirical evidence is still scarce and haphazard. To stimulate further discussion and research, we propose a conceptual framework that describes how environmental interventions are likely to affect human emotional responses. This framework leads to some critical research questions that suggest opportunities for further investigation.

show abstract

“…Among other senses (i.e., audition, touch, taste, and olfaction), auditory stimuli also profoundly provoke human emotions (Redondo, Fraga, Padrón, & Piñeiro, 2008). However, research on auditory system is much less frequent than on visual system (Gerdes et al, 2014). One of the potential reasons would be that visual stimuli are easier to manipulate and control as compared with auditory stimuli (Bradley & Lang, 2000).…”

mentioning

confidence: 99%

“…Studies that are based on these artificial databases may fail to accurately predict the human emotion process, because the sounds we hear in daily life are much more extensive. Therefore, inventing a standardized natural emotional auditory stimulus database that contains sufficient stimuli for emotional research is urgently required (Gerdes et al, 2014).…”

mentioning

confidence: 99%

Affective auditory stimulus database: An expanded version of the International Affective Digitized Sounds (IADS-E)

et al. 2018

View full text Add to dashboard Cite

Using appropriate stimuli to evoke emotions is especially important for researching emotion. Psychologists have provided several standardized affective stimulus databases-such as the International Affective Picture System (IAPS) and the Nencki Affective Picture System (NAPS) as visual stimulus databases, as well as the International Affective Digitized Sounds (IADS) and the Montreal Affective Voices as auditory stimulus databases for emotional experiments. However, considering the limitations of the existing auditory stimulus database studies, research using auditory stimuli is relatively limited compared with the studies using visual stimuli. First, the number of sample sounds is limited, making it difficult to equate across emotional conditions and semantic categories. Second, some artificially created materials (music or human voice) may fail to accurately drive the intended emotional processes. Our principal aim was to expand existing auditory affective sample database to sufficiently cover natural sounds. We asked 207 participants to rate 935 sounds (including the sounds from the IADS-2) using the Self-Assessment Manikin (SAM) and three basic-emotion rating scales. The results showed that emotions in sounds can be distinguished on the affective rating scales, and the stability of the evaluations of sounds revealed that we have successfully provided a larger corpus of natural, emotionally evocative auditory stimuli, covering a wide range of semantic categories. Our expanded, standardized sound sample database may promote a wide range of research in auditory systems and the possible interactions with other sensory modalities, encouraging direct reliable comparisons of outcomes from different researchers in the field of psychology.

show abstract

Emotional pictures and sounds: a review of multimodal interactions of emotion cues in multiple domains

Cited by 91 publications

References 120 publications

Simple and complex crossmodal correspondences involving audition

Simple and complex crossmodal correspondences involving audition

Emotional Responses to Multisensory Environmental Stimuli

Affective auditory stimulus database: An expanded version of the International Affective Digitized Sounds (IADS-E)

Contact Info

Product

Resources

About