Deep Learning of Human Perception in Audio Event Classification

Yu, Yi; Beuret, Samuel; Zeng, Donghuo; Oyama, Keizo

doi:10.1109/ism.2018.00-11

Cited by 14 publications

(7 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 3 shows that our model is trained on EEG responses with the longest stimuli by a large margin on the order of minutes. Other studies have also attempted to not include feature extraction steps but the stimuli length in their experiments are significantly smaller than ours (in the order of seconds) [13,15]. In comparison to [16], the EEG responses in NMED-T were to unfamiliar stimuli, which in prior works has shown to be the harder case as classification performance drops when listeners are not familiar to the music stimuli [23].…”

Section: Comparisonsmentioning

confidence: 94%

“…Another study used longer stimuli (∼10 sec) of 8 varying types of vocalizations and was able to achieve ∼61% performance without any feature extraction on EEG passed to DenseNet. Yu et al [15] improved the performance to ∼81% by incorporating canonical correlation analysis between DenseNet and pre-trained VGG model that extracted audio features of the experimental stimuli. Most recently, Sonawane et al [16] improved on these approaches and showed that longer and complex stimuli ( 2 mins of music) could be used to evoke EEG responses used as spectral 2D CNN inputs to classify song ID (∼85.0%).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Image-based eeg classification of brain responses to song recordings

Ramirez-Aristizabal¹,

Ebrahimpour²,

Kello³

2022

Preprint

View full text Add to dashboard Cite

Classifying EEG responses to naturalistic acoustic stimuli is of theoretical and practical importance, but standard approaches are limited by processing individual channels separately on very short sound segments (a few seconds or less). Recent developments have shown classification for music stimuli (∼ 2 mins) by extracting spectral components from EEG and using convolutional neural networks (CNNs). This paper proposes an efficient method to map raw EEG signals to individual songs listened for end-to-end classification. EEG channels are treated as a dimension of a [Channel × Sample] image tile, and images are classified using CNNs. Our experimental results ( 88.7%) compete with state-of-the-art methods (85.0%), yet our classification task is more challenging by processing longer stimuli that were similar to each other in perceptual quality, and were unfamiliar to participants. We also adopt a transfer learning scheme using a pre-trained ResNet-50, confirming the effectiveness of transfer learning despite image domains being unrelated from each other.

show abstract

Section: Comparisonsmentioning

confidence: 94%

Section: Introductionmentioning

confidence: 99%

Image-based eeg classification of brain responses to song recordings

Ramirez-Aristizabal¹,

Ebrahimpour²,

Kello³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Yi Yu et al used a convolution neural network called DenseNet [16] for audio event classification [7]. The EEG responses were collected on 9 male participants.…”

Section: Related Workmentioning

confidence: 99%

“…They have used engineered features for processing the EEG data which are dependent on the domain knowledge. There have been few attempts on automatic feature extraction from EEG data using neural networks for song classification task [7].…”

Section: Introductionmentioning

confidence: 99%

GuessTheMusic: Song Identification from Electroencephalography response

Sonawane¹,

Miyapuram²,

RS³

et al. 2020

Preprint

View full text Add to dashboard Cite

The music signal comprises of different features like rhythm, timbre, melody, harmony. Its impact on the human brain has been an exciting research topic for the past several decades. Electroencephalography (EEG) signal enables non-invasive measurement of brain activity. Leveraging the recent advancements in deep learning, we proposed a novel approach for song identification using a Convolution Neural network given the electroencephalography (EEG) responses. We recorded the EEG signals from a group of 20 participants while listening to a set of 12 song clips, each of approximately 2 minutes, that were presented in random order. The repeating nature of Music is captured by a data slicing approach considering brain signals of 1 second duration as representative of each song clip. More specifically, we predict the song corresponding to one second of EEG data when given as input rather than a complete two-minute response. We have also discussed pre-processing steps to handle large dimensions of a dataset and various CNN architectures. For all the experiments, we have considered each participant's EEG response for each song in both train and test data. We have obtained 84.96% accuracy for the same. The performance observed gives appropriate implication towards the notion that listening to a song creates specific patterns in the brain, and these patterns vary from person to person.

show abstract

“…In addition, we obtain visual features 𝒗 𝑛 ∈ R 𝑑 𝑣 by applying a simple dimension reduction approach, principal component analysis (PCA), to the obtained features in order to prevent over-fitting. Note that application of PCA to CNN features is generally used for dimension reduction [14,41].…”

Section: Heterogeneous Feature Extractionmentioning

confidence: 99%

Deterioration level estimation via neural network maximizing category-based ordinally supervised multi-view canonical correlation

Maeda

Takahashi

Ogawa

et al. 2020

Multimed Tools Appl

View full text Add to dashboard Cite

A deterioration level estimation method via neural network maximizing category-based ordinally supervised multi-view canonical correlation is presented in this paper. This paper focuses on real world data such as industrial applications and has two contributions. First, a novel neural network handling multi-modal features transforms original features into features effectively representing deterioration levels in transmission towers, which are one of the infrastructures, with consideration of only correlation maximization. It can be realized by setting projection matrices maximizing correlations between multiple features into weights of hidden layers. That is, since the proposed network has only a few hidden layers, it can be trained from a small amount of training data. Second, since there exist diverse characteristics and an ordinal scale in deterioration levels, the proposed method newly derives category-based ordinally supervised multi-view canonical correlation analysis (Co-sMVCCA). Co-sMVCCA enables estimation of effective projection considering both within-class divergence and the ordinal scale between classes. Experimental results showed that the proposed method realizes accurate deterioration level estimation.

show abstract

Deep Learning of Human Perception in Audio Event Classification

Cited by 14 publications

References 9 publications

Image-based eeg classification of brain responses to song recordings

Image-based eeg classification of brain responses to song recordings

GuessTheMusic: Song Identification from Electroencephalography response

Deterioration level estimation via neural network maximizing category-based ordinally supervised multi-view canonical correlation

Contact Info

Product

Resources

About