Enhanced global and local face feature extraction for effective recognition of facial emotions

Abstract

Emotion recognition is a challenging task in the field of human computer interaction.For a successful human emotion recognition system, a robust, discriminative, and sensitive feature extraction is an essential need. In this article, the extraction of global features is done by the proposed frequency decoded lifting wavelet pattern descriptor (FDLWP) and extraction of local features is done by the proposed local gradient difference zig-zag pattern descriptor (LGDZP). The face parts are detected using viola jones algorithm and the selection of optimal face active regions is accomplished by the calculation of structural similarity index measure. Eventually the local spatial zig-zag structure of the face region is utilized to attain LGDZP descriptor. The fusion of local and global features is accomplished using canonical correlation analysis. The classification made using bank of restricted Boltzmann machine (RBM) classifiers yields promising results for the proposed method. Furthermore, the proposed recognition method delivers a promising accuracy in varying illumination, occlusion, and noise. The accuracy of the method is analyzed by doing experiments with the databases such as JAFFE, CK+, MMI, Oulu-CASIA, and SFEW. The obtained results of the proposed method yield better accuracy than the existing state of art methods in this field. K E Y W O R D Scanonical correlation analysis (CCA), facial emotion recognition, frequency decoded lifting wavelet pattern descriptor (FDLWP), local gradient difference zigzag pattern descriptor (LGDZP), restricted Boltzmann machine (RBM), zig zag pattern descriptor INTRODUCTIONHuman emotion recognition is very essential for machines while making customized interactions with a human in human-computer interaction.Humans could get better assistance from machines only if machines understand human emotion. Emotions can be expressed by human verbally and non-verbally. Facial expression is a very common natural way to express the emotion by a human to observer. Normally 55% of human emotions are expressed through facial expressions, 38% through voice, and 7% through spoken words. 1 Retrieving emotion information from face image has come across a wide range of application such as biometric identification, behavioral analysis, stress evaluation, sentiment analysis, intelligent learning, human computer interaction, automated access control and surveillance. 2,3 The motion of face muscles beneath the skin plays a vital role in facial expression, and it is concluded by various researches that, facial expressions conveys emotion well than any other means. 4 Ekman and Friesen suggested six emotions as universal which includes anger, fear, disgust, happy, sad, and surprise. 5 Recent researches in the field of affect computing have made the facial expression recognition a feasible resource for intelligent interactive applications. 6 An automated facial expression recognition system normally has four stages, namely preprocessing and face detection, feature extraction from detected face, dimension red...