Feature Decomposition and Reconstruction Learning for Effective Facial Expression Recognition

Facial Expression Recognition Based on Squeeze Vision Transformer

“…As shown in the experimental results listed in Table 3 , the proposed Squeeze ViT method showed the best performance on the CK+ and MMI datasets. In addition, FDRL [ 37 ] showed a similar performance as the proposed method on CK+ but a 4.66% lower performance on MMI. However, for the wild dataset RAF-DB, FDRL [ 37 ] showed a 0.57% higher performance than the proposed method.…”

“…In addition, FDRL [ 37 ] showed a similar performance as the proposed method on CK+ but a 4.66% lower performance on MMI. However, for the wild dataset RAF-DB, FDRL [ 37 ] showed a 0.57% higher performance than the proposed method. It was determined that the recognition performance of FDRL [ 37 ] was improved because the additional MS-Celeb-1M face recognition database [ 41 ] was used for the pre-training of ResNet-18, the backbone network of FDRL.…”

“…In addition, the proposed method showed an excellent performance on both the wild dataset and the lab-controlled datasets in comparison with the other SOTA FER methods. The proposed method showed a slightly lower performance than the FDRL [ 37 ] method for RAF-DB, which is due to RAF-DB being a wild dataset; thus, the landmarks could not be accurately extracted owing to extreme lighting conditions, occlusions, and severe changes in pose. Therefore, in a future study, we plan to expand this research work to include the extraction of features by estimating landmarks or facial patches even in occluded face regions.…”

“…The SOTA methods used in the comparative experiment are as follows: (1) a hierarchical weighted random forest (WRF)-based FER classifier [ 8 ], (2) inconsistent pseudo annotations to latent truth (IPA2LT) [ 34 ], (3) facial motion prior networks (FMPN) [ 35 ], (4) auxiliary label space graphs (ALSG) [ 36 ], (5) feature decomposition and reconstruction learning (FDRL) [ 37 ], (6) local non-local joint network (LNLAttenNet) [ 38 ], (7) vision transformer jointly with squeeze and excitation (ViT-SE) [ 23 ], (8) distribution mining and pairwise uncertainty estimation (DMUE) [ 39 ], (9) relative uncertainty learning (RUE) [ 40 ], and (10) the proposed Squeeze ViT algorithm.…”

“…However, for the wild dataset RAF-DB, FDRL [ 37 ] showed a 0.57% higher performance than the proposed method. It was determined that the recognition performance of FDRL [ 37 ] was improved because the additional MS-Celeb-1M face recognition database [ 41 ] was used for the pre-training of ResNet-18, the backbone network of FDRL. WRF, a traditional machine learning technique, achieved a relatively poor performance compared with the CNN-based FER methods.…”