This paper presents a mobile application for real time facial expression recognition running on a smart phone with a camera. The proposed system uses a set of Support Vector Machines (SVMs) for classifying 6 basic emotions and neutral expression along with checking mouth status. The facial expression features for emotion recognition are extracted by Active Shape Model (ASM) fitting landmarks on a face and then dynamic features are generated by the displacement between neutral and expression features. We show experimental results with 86% of accuracy with 10 folds cross validation in 309 video samples of the extended Cohn-Kanade (CK+) dataset. Using the same SVM models, the mobile app is running on Samsung Galaxy S3 with 2.4 fps. The accuracy of real-time mobile emotion recognition is about 72% for 6 posed basic emotions and neutral expression by 7 subjects who are not professional actors.
Temporal segmentation of real time video is an important part for automatic facial expression recognition system. Many studies for facial expression recognition have been carried out under restricted experimental environment such as pre-segmented video set. In this paper, we present a real-time temporal video segmenting approach for automatic facial expression recognition applicable in a smartphone. The proposed system uses a Finite State Machine (FSM) for segmenting real time video into temporal phases from neutral expression to the peak of an expression. The FSM uses Lucas-Kanade's optical flow vector based scores for state transitions to adapt the varying speeds of facial expressions. While even HMM based or hybrid HMM model based approaches handling time series data require sampling times, the proposed system runs without any sampling time delay. The proposed system performs facial expression recognition with Support Vector Machines (SVM) on every apex state after automatic temporal segmentation. The mobile app with our approach runs on Samsung Galaxy S3 with 3.7 fps and the accuracy of real-time mobile emotion recognition is about 70.6% for 6 basic emotions by 5 subjects who are not professional actors.
Human motion recognition in video data has several interesting applications in fields such as gaming, senior/assisted-living environments, and surveillance. In these scenarios, we may have to consider adding new motion classes (i.e., new types of human motions to be recognized), as well as new training data (e.g., for handling different type of subjects). Hence, both the accuracy of classification and training time for the machine learning algorithms become important performance parameters in these cases. In this article, we propose a knowledge-based hybrid (KBH) method that can compute the probabilities for hidden Markov models (HMMs) associated with different human motion classes. This computation is facilitated by appropriately mixing features from two different media types (3D motion capture and 2D video). We conducted a variety of experiments comparing the proposed KBH for HMMs and the traditional Baum-Welch algorithms. With the advantage of computing the HMM parameter in a noniterative manner, the KBH method outperforms the Baum-Welch algorithm both in terms of accuracy as well as in reduced training time. Moreover, we show in additional experiments that the KBH method also outperforms the linear support vector machine (SVM).
ACM Reference Format:Suk, M., Ramadass, A., Jin, Y., and Prabhakaran, B. 2012. Video human motion recognition using a knowledge-based hybrid method based on a hidden Markov model.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.