Fusion of spatially separated vowel formant cues

Abstract

Previous studies on fusion in speech perception have demonstrated the ability of the human auditory system to group separate components of speech-like sounds together and consequently to enable the identification of speech despite the spatial separation between the components. Typically, the spatial separation has been implemented using headphone reproduction where the different components evoke auditory images at different lateral positions. In the present study, a multichannel loudspeaker system was used to investigate whether the correct vowel is identified and whether two auditory events are perceived when a noise-excited vowel is divided into two components that are spatially separated. The two components consisted of the even and odd formants. Both the amount of spatial separation between the components and the directions of the components were varied. Neither the spatial separation nor the directions of the components affected the vowel identification. Interestingly, an additional auditory event not associated with any vowel was perceived at the same time when the components were presented symmetrically in front of the listener. In such scenarios, the vowel was perceived from the direction of the odd formant components.

Neural realignment of spatially separated sound components

“…A psychoacoustical experiment was performed to confirm that similar results are obtained using headphone reproduction as in a previous experiment conducted in freefield conditions (Takanen et al, 2013). Specifically, the aim was to verify that the vowel /ae/ is correctly identified despite the spatial separation between the components, and that neither the odd nor the even component alone is sufficient for identification.…”

“…Yet, spatial location has relatively little influence on the grouping of concurrent sounds (Shackleton and Meddis, 1992;Shackleton et al, 1994;Culling and Summerfield, 1995;Darwin, 2008;Schwartz et al, 2012). A spatial separation between two sound sources does not prevent them from being perceptually fused into a single object when other cues are in favor of grouping (Hukin and Darwin, 1995;Hulkin, 1998, 1999;Shinn-Cunningham et al, 2007;Takanen et al, 2013). This fusion makes the identification and localization of concurrently presented sounds a challenging task (Woods and Colburn, 1992;Best et al, 2005;Best et al, 2007;Lee et al, 2009;Croghan and Grantham, 2010;Schwartz et al, 2012).…”

“…1). These spatially separated vowels are perceived as a vowel sound originating from a single location, showing that the components are perceptually placed into the same spatial location (Takanen et al, 2013). The present experiment aimed to reveal how this perceptual reorganization is reflected in the cortical representation of the sound components.…”

“…In contrast, if a single tone is mistuned or it has a different onset time than the rest of the complex, it contributes less to the perceived identity. As another example, a vowel sound can be divided into two components: one containing the odd and the other containing the even-formant components (Takanen et al, 2013), the resonances that determine vowel identity. Neither component alone is sufficient for vowel identification.…”

“…That is, the auditory cortex would represent all sound components belonging to the same auditory object within the same population of spatially selective neurons, regardless of their physical locations. For instance, the spatially separated odd-and even-formant components of a vowel sound (Takanen et al, 2013) would be treated in the brain so that the representations of the individual components would be shifted to match the perceived location of the fused vowel object. Then, from the point of view of the auditory cortex, there would be no more spatial conflict and the identification and localization of the resulting vowel would be straightforward.…”