Facial expression recognition via region-based convolutional fusion network

Ye, Yingsheng; Zhang, Xingming; Lin, Yubei; Wang, Haoxiang

doi:10.1016/j.jvcir.2019.04.009

Cited by 30 publications

(13 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As one of the popular medical imaging technology, CT plays an important role in the COVID-19 diagnosis. Inspired by the success of deep learning in image classification [28] , [29] , deep learning techniques, such as CNN, have been introduced into the field of medical image analysis and have received great interest from the community [30] – [32] .…”

Section: Related Workmentioning

confidence: 99%

Cohesive Multi-Modality Feature Learning and Fusion for COVID-19 Patient Severity Prediction

Zhou

Zhang

Zhu

et al. 2022

IEEE Trans. Circuits Syst. Video Technol.

Self Cite

View full text Add to dashboard Cite

The outbreak of coronavirus disease has been a nightmare to citizens, hospitals, healthcare practitioners, and the economy in 2020. The overwhelming number of confirmed cases and suspected cases put forward an unprecedented challenge to the hospital's capacity of management and medical resource distribution. To reduce the possibility of crossinfection and attend a patient according to his severity level, expertly diagnosis and sophisticated medical examinations are often required but hard to fulfil during a pandemic. To facilitate the assessment of a patient's severity, this paper proposes a multi-modality feature learning and fusion model for end-toend covid patient severity prediction using the blood test supported electronic medical record (EMR) and chest computerized tomography (CT) scan images. To evaluate a patient's severity by the co-occurrence of salient clinical features, the High-order Factorization Network (HoFN) is proposed to learn the impact of a set of clinical features without tedious feature engineering. On the other hand, an attention-based deep convolutional neural network (CNN) using pre-trained parameters are used to process the lung CT images. Finally, to achieve cohesion of cross-modality representation, we design a loss function to shift deep features of both-modality into the same feature space which improves the model's performance and robustness when one modality is absent. Experimental results demonstrate that the proposed multi-modality feature learning and fusion model achieves high performance in an authentic scenario.

show abstract

Section: Related Workmentioning

confidence: 99%

Cohesive Multi-Modality Feature Learning and Fusion for COVID-19 Patient Severity Prediction

Zhou

Zhang

Zhu

et al. 2022

IEEE Trans. Circuits Syst. Video Technol.

Self Cite

View full text Add to dashboard Cite

show abstract

“…And the and technique achieved state-of-art results. Y. Ye et al [24], proposed facial expression recognition method named Region-based Convolutional Fusion Network (CMR). The models built based on VGG model [44].…”

Section: Related Workmentioning

confidence: 99%

Feature Extraction and Fusion for Face Recognition Systems using Pre-Trained Convolutional Neural Networks

Almabdy¹,

Elrefaei²

2021

IJCDS

View full text Add to dashboard Cite

“…In the decoding part, two layers of RBM are developed to reconstruct the input features, so as to form a deep automatic encoder. The weights between the network connection layers are 1 2 3 3 1 2 , , , , , In the process of emotion classification, each channel involved in emotion recognition will eventually get a result of emotion recognition [39]. At this time, each channel can be regarded as a set of separate EEG signals, which can form a separate LIBSVM classifier.…”

Section: B Construction Of Multimodal Emotion Recognition Modelmentioning

confidence: 99%

“…LIBSVM is usually used to solve binary classification problems. Based on the established hyperplane, it can distinguish positive examples and negative examples as much as possible.In the process of emotion classification, each channel involved in emotion recognition will eventually get a result of emotion recognition[39]. At this time, each channel can be regarded as a set of separate EEG signals, which can form a separate LIBSVM classifier.…”

mentioning

confidence: 99%

Expression-EEG Based Collaborative Multimodal Emotion Recognition Using Deep AutoEncoder

Zhang

2020

IEEE Access

View full text Add to dashboard Cite

Emotion recognition has shown many valuable roles in people's lives under the background of artificial intelligence technology. However, most existing emotion recognition methods have poor recognition performance, which prevents their promotion in practical applications. To alleviate this problem, we proposed an expression-EEG interaction multi-modal emotion recognition method using a deep automatic encoder. Firstly, decision tree is applied as objective feature selection method. Then, based on the facial expression features recognized by sparse representation, the solution vector coefficients are analyzed to determine the facial expression category of the test samples. After that, the bimodal deep automatic encoder is adopted to fuse the EEG signals and facial expression signals. The third layer of BDAE extracts features for training of supervised learning. Finally, LIBSVM classifier is used to complete classification task. We carried out experiments on a constructed video library to verify the proposed emotion recognition method. The results show that the proposed method can effectively extract and integrate high-level emotion-related features in EEG and facial expression signals. The recognition rate of discrete emotion state type and the average emotion recognition rate have been improved relatively, in which the average emotion recognition rate is 85.71%. Overall, the emotion recognition ability has been greatly improved.

show abstract

Facial expression recognition via region-based convolutional fusion network

Cited by 30 publications

References 35 publications

Cohesive Multi-Modality Feature Learning and Fusion for COVID-19 Patient Severity Prediction

Cohesive Multi-Modality Feature Learning and Fusion for COVID-19 Patient Severity Prediction

Feature Extraction and Fusion for Face Recognition Systems using Pre-Trained Convolutional Neural Networks

Expression-EEG Based Collaborative Multimodal Emotion Recognition Using Deep AutoEncoder

Contact Info

Product

Resources

About