Autonomous Robot Control Using Facial Expressions

Abstract

Abstract-In this paper, a novel real time robot control system using human facial expressions is presented. The proposed system mainly consists of three modules: face detection, facial expression recognition and robot control. The first module aims to find the user's face from the image captured from a live video through a series of steps including skin color classification, edge detection and mathematical morphology, while the second module analyses the detected face to recognize the facial expressions like happiness, sadness, surprise, anger and neutral using Principal Component Analysis (PCA) and Euclidian distance calculation. Finally, the detected facial expressions are used as controlling commands for the robot. Experiments were conducted for respective modules. 150 images under different lightening conditions and complex background were employed. In the experiment on detecting the facial region, the rate of detection was 99.3% and on recognizing the facial expression, the recognition rate was 97.3%. The robot was then controlled autonomously with an accuracy of 100%. Combining the above three rates, the overall rate of success of our system was 97.3%.

Facial Expressions Recognition: Development and Application to HMI

“…Autonomous robot control systems are complex systems that consist of a sensor, a decision-making control system and a motor drive system. The sensor can be either a visual system, a speech system, or a manual control system [1]. In [2], [3], numerous systems are proposed to control a robot or a wheelchair using head or face movement.…”

Facial expression classification using EEG and gyroscope signals

“…There are a number of inherent downsides to using image processing, namely the need for a camera facing the user's face any time the system wants to be used. Facial expression detection through image processing also has limited functionality if light and pose are suboptimal [10].…”

“…Currently facial expressions can be detected through methods such as image processing [10] or EMG [11]. There are a number of inherent downsides to using image processing, namely the need for a camera facing the user's face any time the system wants to be used.…”