Affective Processes: stochastic modelling of temporal context for emotion and facial expression recognition

Are 3D Face Shapes Expressive Enough for Recognising Continuous Emotions and Action Unit Intensities?

“…When implementing this backbone CNN, we flatten the output feature maps of its last convolutional layer into 2056-dimensional feature vectors, further mapped to 512-dimensional vectors (using an additional fully connected layer) and used as facial embeddings. Considering the relatively small-scale training datasets for AU intensity estimation tasks, we use a ResNet-18 architecture in line with previous works [12], [42], [45].…”

“…EmoFAN [11], [45], a recently proposed 2D CNN model, is designed for facial feature extraction using only convolution layers to make the model more efficient in the number of trainable parameters. A pre-trained variant of this backbone on 2D face alignment tasks is found to be very effective for transfer learning [11], [12].…”

“…To extract the facial features with better generalisation capacity, we use a variant of this CNN backbone pre-trained on image-based emotion recognition using the AffectNet dataset [46], in addition to the 2D face alignment task. Following prior works [11], [45], we use this backbone to extract 512-dimensional facial embedding vectors.…”

“…To compare with the state-of-the-art results on DISFA, following the existing works (e.g. [12], [42], [45]), we also perform the same 3-fold cross-validation; each fold containing 18 videos for training and 9 videos for evaluation.…”

“…To train the dimensional emotion recognition models, we use inverse-CCC + MSE loss, following the objective function originally proposed in [48]. Whereas for the AU intensity estimation, we use MSE alone as the loss function, similar to the existing methods [12], [45]. In both cases, the per-frame loss is accumulated over an input image sequence in computing the total loss per mini-batch.…”