Recognition of Meaningful Human Actions for Video Annotation Using EEG Based User Responses

Moon, Jinyoung; Kwon, Yongjin; Kang, Kyuchang; Bae, Changseok; Yoon, Wan Chul

doi:10.1007/978-3-319-14442-9_50

Cited by 6 publications

(3 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Large amounts of content information and varied visual qualities are packed into multimedia data that are commonly employed in the collection and analysis of EEG data [34][35][36]. Through the study of EEG signals, researchers attempted to determine and categorize the content information of users' watched multimedia material [37][38][39].…”

Section: Literature Surveymentioning

confidence: 99%

Deep Learning Techniques for Pattern Recognition in EEG Audio Signal-Processing-Based Eye-Closed and Eye-Open Cases

Almukhtar¹,

Ajwad

Kamil

et al. 2022

Electronics

View full text Add to dashboard Cite

Recently, pattern recognition in audio signal processing using electroencephalography (EEG) has attracted significant attention. Changes in eye cases (open or closed) are reflected in distinct patterns in EEG data, gathered across a range of cases and actions. Therefore, the accuracy of extracting other information from these signals depends significantly on the prediction of the eye case during the acquisition of EEG signals. In this paper, we use deep learning vector quantization (DLVQ), and feedforward artificial neural network (F-FANN) techniques to recognize the case of the eye. The DLVQ is superior to traditional VQ in classification issues due to its ability to learn a code-constrained codebook. On initialization by the k-means VQ approach, the DLVQ shows very promising performance when tested on an EEG-audio information retrieval task, while F-FANN classifies EEG-audio signals of eye state as open or closed. The DLVQ model achieves higher classification accuracy, higher F score, precision, and recall, as well as superior classification abilities as compared to the F-FANN.

show abstract

Section: Literature Surveymentioning

confidence: 99%

Deep Learning Techniques for Pattern Recognition in EEG Audio Signal-Processing-Based Eye-Closed and Eye-Open Cases

Almukhtar¹,

Ajwad

Kamil

et al. 2022

Electronics

View full text Add to dashboard Cite

show abstract

“…Multimedia data, which contain a large amount of content information and rich visual characteristics, are considered to be a very suitable stimuli material and widely used in the acquisition and analysis of EEG signals [9, 18, 51]. Researchers tried to identify and classify the content information of multimedia data viewed by users through the analysis of EEG signals [15, 52, 53]. Spampinato et al [18], used LSTM network to learn an EEG data representation based on image stimuli and constructed a mapping relationship from natural image features to EEG representation.…”

Section: Related Workmentioning

confidence: 99%

EEG-based image classification via a region-level stacked bi-directional deep learning framework

Fares

Zhong

Jiang

2019

BMC Med Inform Decis Mak

View full text Add to dashboard Cite

BackgroundAs a physiological signal, EEG data cannot be subjectively changed or hidden. Compared with other physiological signals, EEG signals are directly related to human cortical activities with excellent temporal resolution. After the rapid development of machine learning and artificial intelligence, the analysis and calculation of EEGs has made great progress, leading to a significant boost in performances for content understanding and pattern recognition of brain activities across the areas of both neural science and computer vision. While such an enormous advance has attracted wide range of interests among relevant research communities, EEG-based classification of brain activities evoked by images still demands efforts for further improvement with respect to its accuracy, generalization, and interpretation, yet some characters of human brains have been relatively unexplored.MethodsWe propose a region-level stacked bi-directional deep learning framework for EEG-based image classification. Inspired by the hemispheric lateralization of human brains, we propose to extract additional information at regional level to strengthen and emphasize the differences between two hemispheres. The stacked bi-directional long short-term memories are used to capture the dynamic correlations hidden from both the past and the future to the current state in EEG sequences.ResultsExtensive experiments are carried out and our results demonstrate the effectiveness of our proposed framework. Compared with the existing state-of-the-arts, our framework achieves outstanding performances in EEG-based classification of brain activities evoked by images. In addition, we find that the signals of Gamma band are not only useful for achieving good performances for EEG-based image classification, but also play a significant role in capturing relationships between the neural activations and the specific emotional states.ConclusionsOur proposed framework provides an improved solution for the problem that, given an image used to stimulate brain activities, we should be able to identify which class the stimuli image comes from by analyzing the EEG signals. The region-level information is extracted to preserve and emphasize the hemispheric lateralization for neural functions or cognitive processes of human brains. Further, stacked bi-directional LSTMs are used to capture the dynamic correlations hidden in EEG data. Extensive experiments on standard EEG-based image classification dataset validate that our framework outperforms the existing state-of-the-arts under various contexts and experimental setups.

show abstract

“…For the preceding years, unceasing research was seeking to comprehend brain manners through EEGs aroused by purposefully designed stimuli for brain-computer interfacing (BCI) [6,18,32], and conclusions in neuropsychology disclose numeral distinct categories can be identified by event-related potential (ERP) recorded via EEG data [2,13]. Additionally, a variety of machine learning paradigms [10,16,29] have also been established prioritizing the enigma of multimedia-evoked brain grasping via attempts of pattern recognition and categorizations, with enhanced results. In this paper, we expand the stride of the current EEG-based brain research and uphold it toward the novel custom of "brain-media", and hence delve into the possibility of enabling people to foresee what we thought rather than what we see.…”

Section: Introductionmentioning

confidence: 99%

Brain-media: A Dual Conditioned and Lateralization Supported GAN (DCLS-GAN) towards Visualization of Image-evoked Brain Activities

Fares

Zhong

Jiang

2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

out and the results show that our proposed deep framework significantly outperforms the representative existing state-of-the-arts under several settings, especially in terms of both visualization and classification of brain responses to the evoked images. For the convenience of research dissemination, we make the source code openly accessible for downloading at GitHub. 1 CCS CONCEPTS • Computing methodologies → Unsupervised learning; Neural networks.

show abstract

Recognition of Meaningful Human Actions for Video Annotation Using EEG Based User Responses

Cited by 6 publications

References 18 publications

Deep Learning Techniques for Pattern Recognition in EEG Audio Signal-Processing-Based Eye-Closed and Eye-Open Cases

Deep Learning Techniques for Pattern Recognition in EEG Audio Signal-Processing-Based Eye-Closed and Eye-Open Cases

EEG-based image classification via a region-level stacked bi-directional deep learning framework

Brain-media: A Dual Conditioned and Lateralization Supported GAN (DCLS-GAN) towards Visualization of Image-evoked Brain Activities

Contact Info

Product

Resources

About