Digital Twin Model: A Real-Time Emotion Recognition System for Personalized Healthcare

Abstract

Emotion recognition (ER) in healthcare has drawn substantial attention owing to recent advancements in machine-learning (ML) and deep-learning (DL) techniques. The ER system, along with a digital twin of a person in real time, will facilitate the monitoring, understanding, and improvement of the physical entity's capabilities, as well as provide constant input to improve quality of life and well-being for personalized healthcare. However, building such ER systems in real time involves technical challenges, such as limited datasets, occlusion and lighting issues, identifying important features, false classification of emotions, and high implementation costs. To resolve this issue, we built a simple, efficient, and adaptable ER system by acquiring and processing images in real time using a web camera. In addition, we propose an end-to-end framework that combines an ER system with a digital twin setup, in which the predicted result can be analyzed and tested prior to providing the best possible personal healthcare treatment before it leads to any life-threatening disease. Our proposed ER system achieved promising results in less training time without compromising the accuracy. Thus, in real time, it will be helpful in healthcare centers to monitor a patient's health condition, early diagnosis of life-threatening diseases, and to obtain the best and most effective treatment for patients during emergencies.

Human Digital Twin in Industry 5.0: A Holistic Approach to Worker Safety and Well-Being through Advanced AI and Emotional Analytics

“…Other papers examine caring for the mental domain [ 15 , 16 , 17 , 18 , 19 , 20 ] using AI technology as these mental disorders can occasionally cause stress-related problems and have a significant impact on safety and productivity. There are also published papers that consider the importance of emotional well-being at work to achieve a safer work environment, like [ 21 , 22 , 23 , 24 , 25 ], which use different mechanisms, such as virtual reality, cameras or facial expression recognition, to gather emotional status information which is analyzed through AI models; however, they do not consider the physical cross-data, suggest a platform to analyze, or expose the users to the possibility so they can be aware of their overall situation.…”

Emotion Recognition in Psychology of Human-robot Interaction

“…Machine learning techniques in emotion recognition are advancing towards real-time and continuous applications (Subramanian et al, 2022). This is particularly important for interactive systems, such as virtual assistants (Agarwal et al, 2022) or human-robot interactions, where timely and accurate emotion recognition is essential (Wang, 2021a).…”

Emotion Recognition for Affective Human Digital Twin by Means of Virtual Reality Enabling Technologies

“…Reference [9] provides a cooperative city DT idea for municipal crisis management, and [10] suggests using HDTs for elderly real-time monitoring, remote diagnosis, virtual surgery training, and health consulting. A system developed by [11] that uses web cameras to capture and interpret real-time images for emotion recognition in an emergency room. The system integrates with a digital twin setup to allow testing and examination of projected results, preventing life-threatening diseases and providing efficient care.…”

“…Humans can understand, distinguish, and evaluate emotions, and our study aims to recognise emotions based on facial expressions and body movements [3], emotion identification has found fruitful ground for applications: humansrobots interaction [15], action tendency, and recently in Digital twin for health-care [10], [11].…”