Effects of musical experience on perception of audiovisual synchrony for speech and music

Behne, Dawn M.; Alm, Magnus; Berg, Aleksander; Engell, Thomas; Foyn, Camilla; Johnsen, Canutte; Srigaran, Thulasy; Torsdottir, Ane E.

doi:10.1121/1.4801060

Cited by 2 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous research showed subtle differences between musicians and non-musicians in AV perception (Musacchia et al, 2008; Lee and Noppeney, 2011; Paraskevopoulos et al, 2012; Proverbio et al, 2016). For example, a perceiver's prior expectations can influence temporal integration, and relevant training can generate more precise temporal predictions (Noppeney and Lee, 2018) leading to higher sensitivity to AV misalignments (for behavioral study see, Behne et al, 2013, for EEG study see, Behne et al, 2017). Another study (Petrini et al, 2009a) has shown that while musicians have a more refined integration window for AV music perception compared to non-musicians, they are also more accurate at predicting an upcoming sound when the visual information is missing in AV music perception (Petrini et al, 2009b).…”

Section: Discussionmentioning

confidence: 99%

“…This prediction by phonetically congruent visual information can modulate early processing of the audio signal (Stekelenburg and Vroomen, 2007; Arnal et al, 2009; Pilling, 2009; Baart et al, 2014; Hsu et al, 2016; Paris et al, 2016a,b, 2017). Insight into the influence of multisensory experiences, such as musical training, is only beginning to unfold (Petrini et al, 2009a,b, 2011; Lee and Noppeney, 2011, 2014; Paraskevopoulos et al, 2012; Behne et al, 2013; Proverbio et al, 2016; Jicol et al, 2018), and how this regulates audiovisual modulation in speech is yet to be understood.…”

Section: Introductionmentioning

confidence: 99%

“…Musical experience provides an attractive model for studying experience-based neural plasticity. Years of musical practice such as playing an instrument can enhance auditory processing (Zatorre et al, 2007; Strait and Kraus, 2014) and practicing a musical instrument offers a rich multimodal experience, integrating different sensory signals, including audio and visual information (Petrini et al, 2009a,b, 2011; Lee and Noppeney, 2011, 2014; Behne et al, 2013; Jicol et al, 2018). For example, a behavioral study by Petrini et al (2009a) showed that drummers, compared to non-musicians, were more sensitive to AV synchronicity for drumming point-light displays and can even perceptually replace missing visual information (Petrini et al, 2009b).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Musical Expertise Affects Audiovisual Speech Perception: Findings From Event-Related Potentials and Inter-trial Phase Coherence

Sorati¹,

Behne²

2019

Front. Psychol.

View full text Add to dashboard Cite

In audiovisual speech perception, visual information from a talker's face during mouth articulation is available before the onset of the corresponding audio speech, and thereby allows the perceiver to use visual information to predict the upcoming audio. This prediction from phonetically congruent visual information modulates audiovisual speech perception and leads to a decrease in N1 and P2 amplitudes and latencies compared to the perception of audio speech alone. Whether audiovisual experience, such as with musical training, influences this prediction is unclear, but if so, may explain some of the variations observed in previous research. The current study addresses whether audiovisual speech perception is affected by musical training, first assessing N1 and P2 event-related potentials (ERPs) and in addition, inter-trial phase coherence (ITPC). Musicians and non-musicians are presented the syllable, /ba/ in audio only (AO), video only (VO), and audiovisual (AV) conditions. With the predictory effect of mouth movement isolated from the AV speech (AV−VO), results showed that, compared to audio speech, both groups have a lower N1 latency and P2 amplitude and latency. Moreover, they also showed lower ITPCs in the delta, theta, and beta bands in audiovisual speech perception. However, musicians showed significant suppression of N1 amplitude and desynchronization in the alpha band in audiovisual speech, not present for non-musicians. Collectively, the current findings indicate that early sensory processing can be modified by musical experience, which in turn can explain some of the variations in previous AV speech perception research.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Musical Expertise Affects Audiovisual Speech Perception: Findings From Event-Related Potentials and Inter-trial Phase Coherence

Sorati¹,

Behne²

2019

Front. Psychol.

View full text Add to dashboard Cite

show abstract

Children’s Sensitivity to Performance Expression and its Relationship to Children’s Empathy

Taher

2022

Music Perception

View full text Add to dashboard Cite

Emotional communication is central to music performance expression and empathy. Research has shown that music activities can enhance empathy in children and that more empathic adults can more accurately recognize and feel performers’ expressive intentions. Nevertheless, little is known about performance expression during childhood and the specific music-related factors affecting empathy development. This paper explores children’s sensitivity to a performer’s expressive or mechanical intentions and its relationship to children’s everyday empathy. Twenty-seven children listened to expressive and mechanical versions of Romantic flute excerpts with and without accompanying video, rating their perceived level of the performer’s expression and their enjoyment of the performance. The results indicate that children recognize performers’ intended expression or lack thereof and enjoy expressive performances more than mechanical ones. Children aged 10–12 recognized performance expression better than those aged 8–9, especially in audiovisual conditions. Children with higher cognitive empathy rated performance expression more in line with their enjoyment of the performance, which was also more concordant with the performer’s expressive intention. The findings support a relationship between music and socio-emotional skills and emphasize the importance of the visual component of music performance for children, an aspect that has received little attention among researchers and educators.

show abstract

Effects of musical experience on perception of audiovisual synchrony for speech and music

Cited by 2 publications

References 0 publications

Musical Expertise Affects Audiovisual Speech Perception: Findings From Event-Related Potentials and Inter-trial Phase Coherence

Musical Expertise Affects Audiovisual Speech Perception: Findings From Event-Related Potentials and Inter-trial Phase Coherence

Children’s Sensitivity to Performance Expression and its Relationship to Children’s Empathy

Contact Info

Product

Resources

About