Identifying fragments of natural speech from the listener's MEG signals

Koskinen, Miika; Viinikanoja, Jaakko; Kurimo, Mikko; Klami, Arto; Kaski, Samuel; Hari, Riitta

doi:10.1002/hbm.22004

Cited by 31 publications

(23 citation statements)

References 45 publications

(56 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…During real-world experiences, finding relevant brain activities elicited by naturalistic and continuous auditory stimulus can be realized by correlating the temporal courses of brain activities and the temporal courses of features of the stimulus [1], [8]. If the correlation is significant, the corresponding brain activity is regarded to be associated with the stimulus [1], [8].…”

Section: F Clustering Selected Spatial Maps Showing Dipolar Activitymentioning

confidence: 99%

“…However, due to the complexity of the human brain, the state under the naturalistic stimuli including music and video has only recently been decoded through functional magnetic resonance imaging (fMRI) [1]- [6] and magnetoencephalography (MEG) [7], [8]. Brain states during real-world experiences, resulting in relatively low signal-to-noise ratio (SNR) in collected data, are in general more complicated to analyze, than those recorded during the resting state or under the controlled and rapidly repeated stimuli.…”

Section: Introductionmentioning

confidence: 99%

“…However, due to the surprisingly high cost in establishing and maintaining relevant laboratories, fMRI and MEG are not as extensively used as EEG, which does limit the extensive study of brain imaging under real-word experiences. Previous studies have shown that the listener's fMRI data can be significantly correlated with the music stimulus [1] and that from the listener's MEG signals fragments of naturalistic speech can be identified [8]. Hence, it would be natural to infer that the listener's ongoing EEG data can be closely associated with an auditory stimulus including speech and music as well.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Linking Brain Responses to Naturalistic Music Through Analysis of Ongoing EEG and Stimulus Features

Cong

Alluri

Nandi

et al. 2013

IEEE Trans. Multimedia

View full text Add to dashboard Cite

This study proposes a novel approach for the analysis of brain responses in the modality of ongoing EEG elicited by the naturalistic and continuous music stimulus. The 512-second long EEG data (recorded with 64 electrodes) are first decomposed into 64 components by independent component analysis (ICA) for each participant. Then, the spatial maps showing dipolar brain activity are selected in terms of the residual dipole variance through a single dipole model in brain imaging, and clustered into a pre-defined number (estimated by the minimum description length) of clusters. Subsequently, the temporal courses of the EEG theta and alpha oscillations of each component for each cluster are produced and correlated with the temporal courses of tonal and rhythmic features of the music. Using this approach, we found that the extracted temporal courses of the theta and alpha oscillations along central and occipital area of scalp in two of the selected clusters significantly correlated with the musical features representing progressions in the rhythmic content of the stimulus. We suggest that this demonstrates that with the proposed approach, we have managed to discover what kinds of brain responses were elicited when Manuscript a participant was listening continuously to the long piece of naturalistic music.

show abstract

Section: F Clustering Selected Spatial Maps Showing Dipolar Activitymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Linking Brain Responses to Naturalistic Music Through Analysis of Ongoing EEG and Stimulus Features

Cong

Alluri

Nandi

et al. 2013

IEEE Trans. Multimedia

View full text Add to dashboard Cite

show abstract

“…In the experiment, the subjects selectively listened to one of two concurrent spoken narratives mixed into a single acoustic channel, answering comprehension questions about the attended spoken narrative after each 1-min stimulus. The neural recordings were obtained using magnetoencephalography (MEG), which is well suited to measure spatially coherent neural activity synchronized to speech rhythms (i.e., the slow temporal modulations that define the speech envelope) (16)(17)(18)(19). Such spatially coherent phase-locked activity is strongly modulated by attention (20-22) and has been hypothesized to play a critical role in grouping acoustic features into auditory objects (3).…”

mentioning

confidence: 99%

Emergence of neural encoding of auditory objects while listening to competing speakers

Ding¹,

Simon

2012

Proc. Natl. Acad. Sci. U.S.A.

740

114

890

View full text Add to dashboard Cite

A visual scene is perceived in terms of visual objects. Similar ideas have been proposed for the analogous case of auditory scene analysis, although their hypothesized neural underpinnings have not yet been established. Here, we address this question by recording from subjects selectively listening to one of two competing speakers, either of different or the same sex, using magnetoencephalography. Individual neural representations are seen for the speech of the two speakers, with each being selectively phase locked to the rhythm of the corresponding speech stream and from which can be exclusively reconstructed the temporal envelope of that speech stream. The neural representation of the attended speech dominates responses (with latency near 100 ms) in posterior auditory cortex. Furthermore, when the intensity of the attended and background speakers is separately varied over an 8-dB range, the neural representation of the attended speech adapts only to the intensity of that speaker but not to the intensity of the background speaker, suggesting an object-level intensity gain control. In summary, these results indicate that concurrent auditory objects, even if spectrotemporally overlapping and not resolvable at the auditory periphery, are neurally encoded individually in auditory cortex and emerge as fundamental representational units for topdown attentional modulation and bottom-up neural adaptation.spectrotemporal response function | reverse correlation | phase locking | selective attention I n a complex auditory scene, humans and other animal species can perceptually detect and recognize individual auditory objects (i.e., the sound arising from a single source), even if strongly overlapping acoustically with sounds from other sources. To accomplish this remarkably difficult task, it has been hypothesized that the auditory system first decomposes the complex auditory scene into separate acoustic features and then binds the features, as appropriate, into auditory objects (1-4). The neural representations of auditory objects, each the collective representation of all the features belonging to the same auditory object, have been hypothesized to emerge in auditory cortex to become fundamental units for high-level cognitive processing (5-7). The process of parsing an auditory scene into auditory objects is computationally complex and cannot as yet be emulated by computer algorithms (8), but it occurs reliably, and often effortlessly, in the human auditory system. For example, in the classic "cocktail party problem," where multiple speakers are talking at the same time (9), human listeners can selectively attend to a chosen target speaker, even if the competing speakers are acoustically more salient (e.g., louder) or perceptually very similar (such as of the same sex) (10).To demonstrate an object-based neural representation that could subserve the robust perception of an auditory object, several key pieces of evidence are needed. The first is to demonstrate neural activity that exclusively represents a single auditory...

show abstract

“…From the acquired MEG signals, natural speech can be identified using classical canonical correlation analysis (Koskinen et al, 2013). Three-dimensional hand movements were also reconstructed using EEG signals (Bradberry et al, 2010).…”

Section: Introductionmentioning

confidence: 99%

Decoding and encoding of visual patterns using magnetoencephalographic data represented in manifolds

Kuo

Chen

et al. 2014

NeuroImage

View full text Add to dashboard Cite

Identifying fragments of natural speech from the listener's MEG signals

Cited by 31 publications

References 45 publications

Linking Brain Responses to Naturalistic Music Through Analysis of Ongoing EEG and Stimulus Features

Linking Brain Responses to Naturalistic Music Through Analysis of Ongoing EEG and Stimulus Features

Emergence of neural encoding of auditory objects while listening to competing speakers

Decoding and encoding of visual patterns using magnetoencephalographic data represented in manifolds

Contact Info

Product

Resources

About