Local Zernike Moment Representation for Facial Affect Recognition

Abstract

In this paper, we propose to use local Zernike Moments (ZMs) for facial affect recognition and introduce a representation scheme based on performing non-linear encoding on ZMs via quantization. Local ZMs provide a useful and compact description of image discontinuities and texture. We demonstrate the use of this ZM-based representation for posed and discrete as well as naturalistic and continuous affect recognition on standard datasets, and show that ZM-based representations outperform well-established alternative approaches for both tasks. To the best of our knowledge, the performance we achieved on CK+ dataset is superior to all results reported to date.

A dynamic framework based on local Zernike moment and motion history image for facial expression recognition

“…Alternative shape representations include the distances between facial landmarks, distance and angle that represent the opening/closing of the eyes and mouth, and groups of points that describe the state of the cheeks. Although it has been shown that shape representation plays a vital role for analysing facial expressions, they have not been exploited to their full potential [12]. Image moments can be categorised into geometric moments, complex moments and orthogonal moments.…”

“…The rotation invariance of Zernike-based facial features is discussed in [9,10]. QLZM is used in [12] for recognising facial expressions. However, ZM has its shortcomings, namely it is a low level histogram representation which ignores the spatial relations (i.e., configure information) among the different facial parts.…”

“…A ZM of a face image I(x, y) consisting of a real and an imaginary components is [12] Z I nm = n + 1 π…”

“…In the field of facial expression recognition, rotation invariant LBP and uniform LBP [11] have also been used to overcome the rotation problem. In [12], Quantised Local Zernike Moment (QLZM) is used to describe the neighbourhood of a face sub-region. The Local Zernike moments have more discriminant power than other image features, e.g., local phase-magnitude histogram(H-LZM), cascaded LZM transformation (H-LZM 2 ) and local binary pattern (LBP) [13].…”

Emotion Understanding Using Multimodal Information Based on Autobiographical Memories for Alzheimer’s Patients

“…Spatial approaches include shape representations, low-level histograms or Gabor representations amongst others. For example, Huang et al [43] proposed a spatial shape representation using groups of three fiducial points (triangular features) as input to a neural network classifier; and Sariyanidi et al presented in [44] a low-level histogram representation using local Zernike moments for emotion recognition based on kNN and SVM classifiers. On the other hand, spatio-temporal approaches get the features from a range of frames within a temporal window, detecting more efficiently emotions that cannot be easily differentiated in spatial approaches.…”