A Hierarchical Separation and Classification Network for Dynamic Micro-Expression Classification

Abstract

<p> Models of seven discrete facial expressions are built on macro-level facial muscle variations for separating distinct affective states. We propose a step-wise Hierarchical Separation and Classification Network (HSCN) that discovers dynamic and continuous macro- and micro-level variations in facial expressions. The HSCN first invokes an unsupervised cosine similarity-based separation method on continuous facial expression data and extracts twenty-one dynamic expression classes from the seven common discrete affective states. Separation between the clusters is then optimised for discovering the macro-level changes in facial muscle activations followed by splitting the upper and lower facial regions for realising and modelling changes pertaining to upper and lower facial muscle activations. A linear discriminant space is developed for clustering the upper and lower facial images on the basis of similar muscular activation patterns. Actual dynamic data and linear discriminant features are mapped for developing a rule-based expert system that would facilitate classification of twenty-one upper and twenty-one lower facial micro-expressions. Using the random forest algorithm, classification accuracies of 76.11\% were observed for dynamic macro-level facial expression classification. A support vector machine provided 73.63\% and 87.68\% accuracies respectively while classifying upper and lower facial micro-expressions. This work provides a novel framework for the dynamic assessment of affective states. Reported methods and results also provide new insight into the dynamic analysis of facial expressions of affective states. </p>