Audiovisual Interval Size Estimation Is Associated with Early Musical Training

Abstract

Although pitch is a fundamental attribute of auditory perception, substantial individual differences exist in our ability to perceive differences in pitch. Little is known about how these individual differences in the auditory modality might affect crossmodal processes such as audiovisual perception. In this study, we asked whether individual differences in pitch perception might affect audiovisual perception, as it relates to age of onset and number of years of musical training. Fifty-seven subjects made subjective ratings of interval size when given point-light displays of audio, visual, and audiovisual stimuli of sung intervals. Audiovisual stimuli were divided into congruent and incongruent (audiovisual-mismatched) stimuli. Participants’ ratings correlated strongly with interval size in audio-only, visual-only, and audiovisual-congruent conditions. In the audiovisual-incongruent condition, ratings correlated more with audio than with visual stimuli, particularly for subjects who had better pitch perception abilities and higher nonverbal IQ scores. To further investigate the effects of age of onset and length of musical training, subjects were divided into musically trained and untrained groups. Results showed that among subjects with musical training, the degree to which participants’ ratings correlated with auditory interval size during incongruent audiovisual perception was correlated with both nonverbal IQ and age of onset of musical training. After partialing out nonverbal IQ, pitch discrimination thresholds were no longer associated with incongruent audio scores, whereas age of onset of musical training remained associated with incongruent audio scores. These findings invite future research on the developmental effects of musical training, particularly those relating to the process of audiovisual perception.

Vowel Content Influences Relative Pitch Perception in Vocal Melodies

“…A number of factors extraneous to fundamental frequency have influence over the perception of relative pitch. For example, the perception of relative pitch can be influenced by pitch register (Russo & Thompson, 2005a), loudness (McDermott, Lehr, & Oxenham, 2008;Thompson, Peter, Olsen, & Stevens, 2012), facial movement (Abel, Li, Russo, Schlaug, & Loui, 2016;Thompson, Graham, & Russo, 2005;Thompson, Russo, & Livingstone, 2010) and brightness (Allen & Oxenham, 2014;McDermott et al, 2008;Russo & Thompson, 2005b). With regard to the latter, note-to-note changes in brightness that are congruent with changes in pitch have been found to lead to an expansion of perceived interval size (i.e., pitch distance), whereas incongruent changes lead to a contraction.…”

Visually induced gains in pitch discrimination: Linking audio-visual processing with auditory abilities

“…Semantic influences may well explain, for example, why musicians in previous studies have shown increased sensitivity to pitch-height congruency (Eitan & Granot, 2006;Paraskevopoulos et al, 2015), possibly owing to their increased familiarity with musical notation. Indeed, many previous studies have used musical notation or closely resembling visual stimuli; and, not surprisingly, in their familiar domain, musicians have shown increased behavioral benefits compared to nonmusicians (Abel, Li, Russo, Schlaug, & Loui, 2016;Nichols & Grahn, 2016;Paraskevopoulos et al, 2015;Paraskevopoulos et al, 2012). It is indeed plausible that considerable experience with musical notation and terminology results in an association between pitch and vertical position that is quite explicit in musicians, and that this familiarity effect becomes apparent whenever the pitch intervals can actually be represented by musical notation, resulting in musicians' stronger associations between vertical position and pitch as found in previous studies.…”

“…It is indeed plausible that considerable experience with musical notation and terminology results in an association between pitch and vertical position that is quite explicit in musicians, and that this familiarity effect becomes apparent whenever the pitch intervals can actually be represented by musical notation, resulting in musicians' stronger associations between vertical position and pitch as found in previous studies. However, owing to the focus on the PoIE, the present study targeted visual influences on responses to near-threshold auditory stimuli (20 and 30 cents deviants) as opposed to musical scale intervals (1+ semitones), where musicians may benefit from their explicit knowledge about pitch intervals and their labels (Abel et al, 2016;Nichols & Grahn, 2016). Therefore, the pitch intervals could not be represented by musical notation in the present study.…”

Can Haptic Stimulation Enhance Music Perception in Hearing-Impaired Listeners?

“…Viewing the performer and reading lyrics can increase their musical enjoyment ( Gfeller et al, 2000 ; Looi and She, 2010 ) and raves targeted at the deaf community frequently include musical visualization. Furthermore, the size of sung musical intervals can be determined when only viewing the singer’s face (without audio), with larger intervals associated with more head movement, eyebrow raising, and mouth opening ( Thompson and Russo, 2007 ; Abel et al, 2016 ). Viewing a singer’s face with accompanying audio can also bias the perception of pitch interval size ( Thompson et al, 2010 ), with the mouth apparently increasing in significance as audio signal-to-noise ratios become more challenging ( Russo et al, 2011 ).…”