Detection of Emotion Using Multi-Block Deep Learning in a Self-Management Interview App

Shin, Dong Hoon; Chung, Kyungyong; Park, Roy C.

doi:10.3390/app9224830

Cited by 15 publications

(14 citation statements)

References 29 publications

(26 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Accordingly, through a detection process module, pre-processing is conducted for dimensionality reduction and learning. To analyze the original data transferred through real-time streaming, input images are segmented at 85 fps, and to increase the recognition rate, the particular facial image sections required for facial expression recognition are extracted using the multi-block method [35]. In particular, in cases where a multi-block is big or small during the blocking process, pre-existing models are unable to accurately extract features from the main areas, and this causes significant errors relating to recognition and learning.…”

Section: Real-time Stream Image Data Pre-processing For Facial Expression Recognition-based Health Risk Extractionmentioning

confidence: 99%

Driver Facial Expression Analysis Using LFA-CRNN-Based Feature Extraction for Health-Risk Decisions

et al. 2020

View full text Add to dashboard Cite

As people communicate with each other, they use gestures and facial expressions as a means to convey and understand emotional state. Non-verbal means of communication are essential to understanding, based on external clues to a person’s emotional state. Recently, active studies have been conducted on the lifecare service of analyzing users’ facial expressions. Yet, rather than a service necessary for everyday life, the service is currently provided only for health care centers or certain medical institutions. It is necessary to conduct studies to prevent accidents that suddenly occur in everyday life and to cope with emergencies. Thus, we propose facial expression analysis using line-segment feature analysis-convolutional recurrent neural network (LFA-CRNN) feature extraction for health-risk assessments of drivers. The purpose of such an analysis is to manage and monitor patients with chronic diseases who are rapidly increasing in number. To prevent automobile accidents and to respond to emergency situations due to acute diseases, we propose a service that monitors a driver’s facial expressions to assess health risks and alert the driver to risk-related matters while driving. To identify health risks, deep learning technology is used to recognize expressions of pain and to determine if a person is in pain while driving. Since the amount of input-image data is large, analyzing facial expressions accurately is difficult for a process with limited resources while providing the service on a real-time basis. Accordingly, a line-segment feature analysis algorithm is proposed to reduce the amount of data, and the LFA-CRNN model was designed for this purpose. Through this model, the severity of a driver’s pain is classified into one of nine types. The LFA-CRNN model consists of one convolution layer that is reshaped and delivered into two bidirectional gated recurrent unit layers. Finally, biometric data are classified through softmax. In addition, to evaluate the performance of LFA-CRNN, the performance was compared through the CRNN and AlexNet Models based on the University of Northern British Columbia and McMaster University (UNBC-McMaster) database.

show abstract

Section: Real-time Stream Image Data Pre-processing For Facial Expression Recognition-based Health Risk Extractionmentioning

confidence: 99%

Driver Facial Expression Analysis Using LFA-CRNN-Based Feature Extraction for Health-Risk Decisions

et al. 2020

View full text Add to dashboard Cite

show abstract

“…LFA makes use of the line-segment information on the facial regions in an analyzed image. To extract a face from a real-time image, we apply the Haar based cascade classifier using multiple AdaBoost [31].…”

Section:  Step 1 Facial Contour Segment Classification and Unique Numbering For Facial Expression Analysismentioning

confidence: 99%

Real-Time Streaming Image Based PP2LFA-CRNN Model for Facial Sentiment Analysis

Kim

Lee

et al. 2020

IEEE Access

View full text Add to dashboard Cite

In modern society, the real-time emotion adaptive driving system for providing safety to drivers and emotion-based services are being researched. However, in the service process have problem of personal information might get leaked. Therefore, a robust personal information protection method is required for face recognition services based on real-time images. In this study, we propose a real-time streaming image based PingPong256 (PP2) algorithm, line-segment feature analysis (LFA), convolutional recurrent neural network (CRNN) model for facial sentiment analysis. The proposed method applied the PP2 algorithm to images for encryption and decryption for the security of the real-time images collected by image devices. For transmitting images to a server, LFA, as a dimensionality reduction algorithm, is used to extract facial information. PP2 encrypts and decrypts an image through a linear feedback shift register with a different length and sets a random value other than 0 so that inferring the initial value of encryption becomes difficult, and then executes the random operations approximately 1,000 times. The LFA analyzes the line segments of an image, assigns a different unique number depending on its type, and cumulatively adds them to generate a Line-Segment map (LS-map) with a size of 16 × 16. The LS-map is used as an input of the CRNN model designed in this study, and the facial expressions are classified. Performance evaluation compares the accuracy of face recognition by using the proposed method with the loss rate for other models. Performance evaluation renders excellence to the accuracy of face recognition and loss rate comparison.

show abstract

“…With this technique, it is possible to model a complex, non-linear relationship. Deep learning adjusts the values of units (the weights) to model complex data by placing a large number of units (or nodes) in each layer [16][17][18]. Such deep learning has been designed as feedforward, but recently, it has been used as a recurrent neural network (RNN) in which a deep structure is converted into a circular structure.…”

Section: Deep Learning Technique For Time Series Analysismentioning

confidence: 99%

Urban traffic accident risk prediction for knowledge-based mobile multimedia service

Park

Hong

2020

Pers Ubiquit Comput

View full text Add to dashboard Cite

Traditional accident prediction models have been mostly designed with statistical analysis that finds and analyzes the causal relationships between traffic accidents and a variety of human, road geometry, and environmental factors. However, these statistical methods have limitations in that they are based on assumptions about data distribution and function type. Therefore, this study suggests an accident prediction model using deep learning. This newly suggested risk prediction model is for predicting risk by reflecting static features of the road, such its length and the speed limit on it, and dynamic features of the road, such as traffic volume when driving on it, and the altitude and azimuth of the sun. For this purpose, 4470 accident cases, collected over 5 months from August to December 2018 in Seoul-the most complex, high-traffic, and accident-prone city in Korea-were analyzed. As a result of testing the model using such data, it was found to have an accuracy of 75% and recall of 81%. Based on testing results for the suggested risk prediction model, a system was developed to guide not only accident-prone regions predicted using statistical data but to also guide a risk level for the road. This level of risk is estimated based upon each given situation, so it can change even for the same road. This guide system can be used to provide a level of risk for each road segment and region but also to improve roads with recommendations, such as installation of safety features. In addition, it could support a mobile system that provides a driver with the optimized driving path for safety.

show abstract

Detection of Emotion Using Multi-Block Deep Learning in a Self-Management Interview App

Cited by 15 publications

References 29 publications

Driver Facial Expression Analysis Using LFA-CRNN-Based Feature Extraction for Health-Risk Decisions

Driver Facial Expression Analysis Using LFA-CRNN-Based Feature Extraction for Health-Risk Decisions

Real-Time Streaming Image Based PP2LFA-CRNN Model for Facial Sentiment Analysis

Urban traffic accident risk prediction for knowledge-based mobile multimedia service

Contact Info

Product

Resources

About