Automated Facial Expression and Speech Emotion Recognition App Development on Smart Phones using Cloud Computing

Alshamsi, Humaid; Këpuska, Veton; Alshamsi, Hazza; Meng, Hongying

doi:10.1109/iemcon.2018.8614831

Cited by 21 publications

(16 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In [39], four pitch and spectral energy features were combined with two prosodic features to distinguish two high activation states of angry and happy plus low activation states of sadness and boredom for speech emotion recognition using SVM with Emo-DB. Alshamsi et al [40] proposed a smart phone method for automated facial expression and speech emotion recognition using SVM with MFCC features extracted from SAVEE database.…”

Section: Related Studiesmentioning

confidence: 99%

“…We carefully applied the specialized jAudio software [55] to construct MFCC1, MFCC2 and HAF as three sets of features representing several voice aspects. MFCC1 is the set of MFCC features inspired by the applications of MFCC features [24,40,42]. MFCC2 is the set of features based on MFCC, energy, ZCR and fundamental frequency as inspired by the fusion of MFCC with other acoustic features [11,25,26,44,46].…”

Section: Feature Extractionmentioning

confidence: 99%

“…MFCC2 is the set of features based on MFCC, energy, ZCR and fundamental frequency as inspired by the fusion of MFCC with other acoustic features [11,25,26,44,46]. The HAF is the proposed set of hybrid acoustic features of prosodic and spectral carefully selected based on the interesting results in the literature [11,20,[24][25][26]29,30,39,40,42,44,46,48].…”

Section: Feature Extractionmentioning

confidence: 99%

See 2 more Smart Citations

Ensemble Learning of Hybrid Acoustic Features for Speech Emotion Recognition

Zvarevashe

Olugbara

2020

Algorithms

View full text Add to dashboard Cite

Automatic recognition of emotion is important for facilitating seamless interactivity between a human being and intelligent robot towards the full realization of a smart society. The methods of signal processing and machine learning are widely applied to recognize human emotions based on features extracted from facial images, video files or speech signals. However, these features were not able to recognize the fear emotion with the same level of precision as other emotions. The authors propose the agglutination of prosodic and spectral features from a group of carefully selected features to realize hybrid acoustic features for improving the task of emotion recognition. Experiments were performed to test the effectiveness of the proposed features extracted from speech files of two public databases and used to train five popular ensemble learning algorithms. Results show that random decision forest ensemble learning of the proposed hybrid acoustic features is highly effective for speech emotion recognition.

show abstract

Section: Related Studiesmentioning

confidence: 99%

Section: Feature Extractionmentioning

confidence: 99%

Section: Feature Extractionmentioning

confidence: 99%

See 1 more Smart Citation

Ensemble Learning of Hybrid Acoustic Features for Speech Emotion Recognition

Zvarevashe

Olugbara

2020

Algorithms

View full text Add to dashboard Cite

show abstract

“…In fact, smartphones and smart watches are equipped with a variety of sensors, which include accelerometer, gyroscope, fingerprint sensor, heart rate sensor, and microphone. Alshamsi et al [37] proposed a framework that consisted of smartphone sensor technology supported by cloud computing for the real-time recognition of emotion in speech and facial expression. Hossain et al [38] combined the potential of emotion-aware big data and cloud technology towards 5G.…”

Section: Visual Signal Based Emotion Classificationmentioning

confidence: 99%

Facial Emotion Recognition Using Hybrid Features

Alreshidi

Ullah

2020

Informatics

View full text Add to dashboard Cite

Facial emotion recognition is a crucial task for human-computer interaction, autonomous vehicles, and a multitude of multimedia applications. In this paper, we propose a modular framework for human facial emotions’ recognition. The framework consists of two machine learning algorithms (for detection and classification) that could be trained offline for real-time applications. Initially, we detect faces in the images by exploring the AdaBoost cascade classifiers. We then extract neighborhood difference features (NDF), which represent the features of a face based on localized appearance information. The NDF models different patterns based on the relationships between neighboring regions themselves instead of considering only intensity information. The study is focused on the seven most important facial expressions that are extensively used in day-to-day life. However, due to the modular design of the framework, it can be extended to classify N number of facial expressions. For facial expression classification, we train a random forest classifier with a latent emotional state that takes care of the mis-/false detection. Additionally, the proposed method is independent of gender and facial skin color for emotion recognition. Moreover, due to the intrinsic design of NDF, the proposed method is illumination and orientation invariant. We evaluate our method on different benchmark datasets and compare it with five reference methods. In terms of accuracy, the proposed method gives 13% and 24% better results than the reference methods on the static facial expressions in the wild (SFEW) and real-world affective faces (RAF) datasets, respectively.

show abstract

“…To this end, smart mobiles and smart wrist watches are fully equipped with different types of sensors, for instance, accelerometer, gyroscope, fingerprint Sensor, heart rate sensor, and microphone. Alshamsi et al [24] investigated a method driven by sensor technology and cloud computing for identification of emotion in both speech and facial expression. Hossain et al [25] introduced a framework that puts together the strengths of emotion-aware big data and cloud technology towards 5G.…”

Section: Literature Reviewmentioning

confidence: 99%

Facial Emotion Recognition using Neighborhood Features

Aljaloud¹,

Ullah²,

Alanazi³

2020

IJACSA

View full text Add to dashboard Cite

We present a new method for human facial emotions recognition. For this purpose, initially, we detect faces in the images by using the famous cascade classifiers. Subsequently, we then extract a localized regional descriptor (LRD) which represents the features of a face based on regional appearance encoding. The LRD formulates and models various spatial regional patterns based on the relationships between local areas themselves instead of considering only raw and unprocessed intensity features of an image. To classify facial emotions into various classes of facial emotions, we train a multiclass support vector machine (M-SVM) classifier which recognizes these emotions during the testing stage. Our proposed method takes into account robust features and is independent of gender and facial skin color for emotion recognition. Moreover, our method is illumination and orientation invariant. We assessed our method on two benchmark datasets and compared it with four reference methods. Our proposed method outperformed them considering both the datasets.

show abstract

Automated Facial Expression and Speech Emotion Recognition App Development on Smart Phones using Cloud Computing

Cited by 21 publications

References 38 publications

Ensemble Learning of Hybrid Acoustic Features for Speech Emotion Recognition

Ensemble Learning of Hybrid Acoustic Features for Speech Emotion Recognition

Facial Emotion Recognition Using Hybrid Features

Facial Emotion Recognition using Neighborhood Features

Contact Info

Product

Resources

About