3D face model dataset: Automatic detection of facial expressions and emotions for educational environments

Abstract

Emotion detection using facial images is a technique that researchers have been using for the last two decades to try to analyze a person's emotional state given his/her image. Detection of various kinds of emotion using facial expressions of students in educational environment is useful in providing insight into the effectiveness of tutoring sessions. Robust recognition with conventional 2D cameras is still not possible under realistic conditions, ie, in the presence of variation in lighting and illumination and pose variations of the participants. This paper discusses in brief how facial expression recognition from 3D models could be achieved for applications under various educational environments. 3D Face Model dataset for various facial expressions of about 95 undergraduate students of the Department of Information Science and Engineering was collected. The new dataset produced comprise 3D models of emotions represented by five groups of facial expressions, namely: normal, happy, sad, surprised and angry, which are mainly seen among students during their college days. These data were collected using a Kinect camera, which also gives the depth map for the facial expressions. The dataset collected is a point cloud. The Kinect camera measures a large number of points on the face of the person and this output is stored as a point cloud data (PCD) file. The point cloud thus represents the sets of points that the camera has measured. The data are stored in the PCD file format, which supports the 3D point cloud data, and which can be viewed using any point cloud library (PCL) viewer. IntroductionThe face, body, posture and action contribute in conveying the emotional state of the individual. These are treated as nonlinguistic symbols, which compliment the linguistic style of the person.

Emotional expression in psychiatric conditions: New technology for clinicians

“…We have tried to include the diagnostic and therapeutic methods that have been published in scientific journals, but the data, including quantitative and qualitative information, are limited, which causes an important drawback of this paper. For the summary of these methods see Table . What we can expect in the future is a rapid growth of use of modern technologies, such as VR, augmented reality, and interactive systems based on game consoles, in therapy enhancement.…”

“…The purpose of this article is to investigate the physical characteristics of emotion expressiveness across these modalities in several psychiatric conditions, and to examine how new technology can support clinical practice. A summary of the most important methods and studies on emotion expression assessment methods is presented in Table . The idea that emotions are innate, unlearned responses associated with a complex set of movements affecting the face, body, and speech traces back to Darwin, who suggested that the way in which we express emotions has evolved from emotional expression in other animals.…”

“…Current techniques also allow us to capture movement in 3‐D and include its dynamics. The hardware required for this is simple, such as a camera used in a game console (Kinect) . In some cases, even simple home cameras or a smartphone have been shown to be sufficient …”

An evaluation of an adaptive learning system based on multimodal affect recognition for learners with intellectual disabilities

“…This may be in terms of different facial expressions, posture and movements due to co‐occurring neurological differences and different use of eye gaze in those with autistic tendencies. Self‐annotation of recordings (eg, Chickerur & Joshi, 2015) was not appropriate in our study due to the ability of the participants. Therefore, rather than use a method established for main stream learners (eg, BROMP, Ocumpaugh, Baker, & Rodrigo, 2012), in order to determine the affective state of students with ID, teachers and trained researchers familiar with the students annotated recordings of them working on educational materials on a variety of platforms (desktop, mobile devices and educational robots).…”

Emotion estimation in crowds: a survey

“…However, as the authors in [23] argue, using conventional 2D cameras lacks robustness as this kind of cameras are subject to poor illumination and changes in lighting conditions. In response, they propose the use of Kinetic cameras as these are able to capture depth, and produce a dataset containing 3D models of several participants performing multiple facial expressions.…”