EmotionNet Nano: An Efficient Deep Convolutional Neural Network Design for Real-Time Facial Expression Recognition

Abstract

While recent advances in deep learning have led to significant improvements in facial expression classification (FEC), a major challenge that remains a bottleneck for the widespread deployment of such systems is their high architectural and computational complexities. This is especially challenging given the operational requirements of various FEC applications, such as safety, marketing, learning, and assistive living, where real-time requirements on low-cost embedded devices is desired. Motivated by this need for a compact, low latency, yet accurate system capable of performing FEC in real-time on low-cost embedded devices, this study proposes EmotionNet Nano, an efficient deep convolutional neural network created through a human-machine collaborative design strategy, where human experience is combined with machine meticulousness and speed in order to craft a deep neural network design catered toward real-time embedded usage. To the best of the author’s knowledge, this is the very first deep neural network architecture for facial expression recognition leveraging machine-driven design exploration in its design process, and exhibits unique architectural characteristics such as high architectural heterogeneity and selective long-range connectivity not seen in previous FEC network architectures. Two different variants of EmotionNet Nano are presented, each with a different trade-off between architectural and computational complexity and accuracy. Experimental results using the CK + facial expression benchmark dataset demonstrate that the proposed EmotionNet Nano networks achieved accuracy comparable to state-of-the-art FEC networks, while requiring significantly fewer parameters. Furthermore, we demonstrate that the proposed EmotionNet Nano networks achieved real-time inference speeds (e.g., >25 FPS and >70 FPS at 15 and 30 W, respectively) and high energy efficiency (e.g., >1.7 images/sec/watt at 15 W) on an ARM embedded processor, thus further illustrating the efficacy of EmotionNet Nano for deployment on embedded devices.

Analyzing the Role of Emotional Intelligence on the Performance of Small and Medium Enterprises (SMEs) Using AI-Based Convolutional Neural Networks (CNNs)

“…e scientists also showed that the proposed EmotionNet Nano networks could attain real-time inference speeds (e.g., >25 FPS and >70 FPS at 15 and 30 W, respectively) and great energy efficiency (e.g., >1.7 images/ sec/watt at 15 W) [42].…”

“…e raw data were then preprocessed to eliminate artefacts such as powerline disturbance, which was eliminated using a 50 Hz bandpass filter, proceeded by a glitter bandpass Butterworth filter to retrieve the essential frequencies in the range of 0.05 to 100 Hz to eliminate noise. In our studies, researchers used a Glitter bandpass Butterworth filter to retrieve data from the signal in four main frequency bands: α (2-8 Hz), β (9-14 Hz), θ (15-31 Hz), and c bands (31)(32)(33)(34)(35)(36)(37)(38)(39)(40)(41)(42)(43)(44)(45)(46).…”

Training and Profiling a Pediatric Facial Expression Classifier for Children on Mobile Devices: Machine Learning Study

“…Although results with CNNs in FER have improved consistently through recent years and CNNs have been used in similar applications such as eye gaze detection [ 68 , 69 ], there are few endeavors involving classification on children’s faces. The CAFE data set [ 49 ] currently is the largest publicly available data set of facial expressions from children and is a standard benchmark in the field of FER on children.…”

Perbandingan AlexNet dan VGG untuk Pengenalan Ekspresi Wajah pada Dataset Kelas Komputasi Lanjut

“…Fitur-fitur tersebut dipilih pada tahap pemilihan fitur, sehingga diperoleh beberapa fitur yang mengandung lebih banyak informasi untuk mengklasifikasikan kelas yang berbeda. Pada langkah terakhir (klasifikasi ekspresi wajah), metode pengklasifikasian berbasis machine learning seperti knearest neighbor [8], support vector machine [9], dan neural network [10] terlebih dulu dilatih, kemudian digunakan untuk mengklasifikasikan ekspresi wajah.…”