Automatic recognition of macaque facial expressions for detection of affective states

Abstract

Internal affective states produce external manifestations such as facial expressions. In humans, the Facial Action Coding System (FACS) is widely used to objectively quantify the elemental facial action-units (AUs) that build complex facial expressions. A similar system has been developed for macaque monkeys - the Macaque Facial Action Coding System (MaqFACS); yet unlike the human counterpart, which is already partially replaced by automatic algorithms, this system still requires labor-intensive coding. Here, we developed and implemented the first prototype for automatic MaqFACS coding. We applied the approach to the analysis of behavioral and neural data recorded from freely interacting macaque monkeys. The method achieved high performance in recognition of six dominant AUs, generalizing between conspecific individuals (Macaca mulatta) and even between species (Macaca fascicularis). The study lays the foundation for fully automated detection of facial expressions in animals, which is crucial for investigating the neural substrates of social and affective states.

Automated Recognition Of Pain In Cats

“…Neither automatic landmarking, nor pain estimation were found directly transferable to donkeys. Morozov et al 40 developed and implemented a prototype system for automatic MaqFACS coding, using a dataset which included 53 videos from 5 Rhesus macaques capturing frontal views of head-fixed individuals; the video-frames were manually coded for the AUs present in each frame. The system was trained to classify six MacFACS AUs reaching average 89% accuracy.…”

Going Deeper than Tracking: a Survey of Computer-Vision Based Recognition of Animal Pain and Affective States

“…An explorative search for behaviors which potentially extends over time might also be desirable, and the degrees of freedom offered by spatiotemporal feature learning approaches is useful for such a task. [85] macaques holistic frame hand-crafted (high-level) Blumrosen et al [86] macaques holistic frame hand-crafted (high-level) Zhu [87] dogs holistic frame mixed Franzoni et al [88] dogs holistic frame learned Boneh-Shitrit et al [89] dogs holistic frame learned Ferres et al [90] dogs emotion frame hand-crafted (high-level) Statham et al [91] pigs emotion frame -…”

“…The column 'Features' in Table 2 classifies the works across the dimension of learned vs. hand-crafted features. The types of high-level features used in [76,[84][85][86]90] are further discussed in Section 4.4.…”

“…• Facial Action Units. Morozov et al [85] and Blumrosen et al [86] apply two different approaches to address the recognition of facial actions in macaques as an intermediate step towards automated analysis of affective states. [85] addresses six dominant action units from macaque FACS (MaqFACS [113]), selected based on their frequency and importance for affective communication, training a classifier on data annotated by human experts.…”

Deep Learning Models for Automated Classification of Dog Emotional States from Facial Expressions

“…Pain recognition from facial expressions has been investigated for rodents [2,42,44], sheep [37], equines [9,29,33], and cats [23]. Action unit recognition was automated for several types of non-human primates [5,39]. In the context of dogs, Ferres et al studied automated pose estimation using DeepLabCut [38] for classification of emotional states as anger, fear, happiness and relaxation [24].…”