The recognition of basic facial expressions by neural network

Abstract

The Recognition of Basic Facial Expressions by Neural NetworkHiroshi KOBAYASHI* and Fumio HARA**We propose the concept of Active Human Interface (AHI) that makes the machine (computer and/or robot) respond to human being more actively and for establishing the new paradigm to realize the AHI, as the first step of this study, we investigate the method of machine recognition of human emotions.This paper deals with the neural network method of human emotion recognition from facial expressions. Facial expressions were categorized into 6 groups (Surprise, Fear, Disgust, Anger, Happiness and Sadness), and obtained CCD camera-acquired data with respect to facial characteristic points relating to 3 components of face (Eyebrows, Eyes and Mouth). Then we generated the position information and shape information about the 6 basic facial expressions for 30 clients. These information were input into the Input units of the 4-layered neural network and network learning was carried out by back propagation algorithm. The neural network recognition system of facial expressions showed a high recognition rate up to 80% to 6 basic facial expressions for both the position and shape information and particularly the system showed a smaller rate of mis-recognition between some of 6 basic expressions.

General and Interval Type-2 Fuzzy Face-Space Approach to Emotion Recognition

“…Their experiments confirmed the existence of a good correlation between basic movements of the facial action units [13], [19] and facial expressions [1], [2], [5], [7], [10], [19]- [22]. Kobayashi and Hara [15]- [17] designed a scheme for the recognition of human facial expressions using the well-known back-propagation neural networks [38], [43]. Their scheme is capable of recognizing six common facial expressions depicting happiness, sadness, fear, anger, surprise, and disgust.…”

“…Gao et al presented a scheme for facial expression recognition from a single facial image using line based caricatures [53]. Among other significant contributions in emotion recognition, the works presented in [6], [8], [9], [11], [12], [15]- [17], [23]- [28], [30]- [32], [35], [40], [46], [56], [57], [60], [70], [72], [77]- [80] need special mention. For a more complete literature survey, which cannot be given here for space restriction, readers may refer to two outstanding papers by Pantic et al [57], [67].…”

Recognition of Six basic facial expression and their strength by neural network

“…For t h e value of "wcak", I f t h e value of "strong" Is 1.0, I s t h e one shown in Tablc n e u r a l network for "weak" Information to t h a t obtained by Inputting "strong" one to t h e n e u r a l network. W e compared t h e ratios with those obtalned in t h e s t r e n g t h recognitlon test by human being and shown i n Table 2 4 indicates a r a t h e r high agreement between t h e r e s u l t s from human being test and t h a t of t h e n e u r a l network test. Moreover, in o r d e r t o e v a l u a t e t h e discrepancy between both r e s u l t s more c l e a r l y , we c a l c u l a t e d t h e s q u a r e sum of discrepancy between both r e s u l t s and t h e results a r e shown in Table 4.…”

Lip shape based emotion identification

“…Kobayashi ve Hara ise yüz ifadelerini tanımak için geriye yayılım algoritması ile çalışan yapay sinir ağlarını kullanmışlardır [12,13,14]. Cohen, Hidden Markov Modelini kullanarak videolardan yüz ifadelerini tespit etmiştir [15].…”