Automated Affect and Emotion Recognition from Cardiovascular Signals - A Systematic Overview Of The Field

Abstract

Currently, artificial intelligence is increasingly used to recognize and differentiate emotions.Through the action of the nervous system, the heart and vascular system can respond differently depending on the type of arousal. With the growing popularity of wearable devices able to measure such signals, people may monitor their states and manage their wellness. Our goal was to explore and summarize the field of automated emotion and affect recognition from cardiovascular signals.According to our protocol, we searched electronic sources (MEDLINE, EMBASE, Web of Science, Scopus, dblp, Cochrane Library, IEEE Explore, arXiv and medRxiv) up to 31 August 2020. In the case of all identified studies, two independent reviewers were involved at each stage: screening, full-text assessment, data extraction, and quality evaluation. All conflicts were resolved during the discussion. The credibility of included studies was evaluated using a proprietary tool based on QUADAS, PROBAST. After screening 4649 references, we identified 195 eligible studies.From artificial intelligence most used methods in emotion or affect recognition were Support Vector Machines (42.86%), Neural Network (21.43%), and k-Nearest Neighbors (11.67%).Among the most explored datasets were DEAP (10.26%), MAHNOB-HCI (10.26%), AMIGOS (6.67%) and DREAMER (2.56%). The most frequent cardiovascular signals involved electrocardiogram (63.16%), photoplethysmogram (15.79%), blood volume pressure (13.16%) and heart rate (6.58%).Sadness, fear, and anger were the most examined emotions.However, there is no standard set of investigated internal feelings. On average, authors explore 4.50 states (range from 4 to 24 feelings).Research using artificial intelligence in recognizing emotions or affect using cardiovascular signals shows an upward trend. There are significant variations in the quality of the datasets, the choice of states to detect, and the classifiers used for analysis.

Datasets for Automated Affect and Emotion Recognition from Cardiovascular Signals Using Artificial Intelligence— A Systematic Review

“…Supplementary File S1 (OSF [ 64 ]) and Appendix A and Appendix B contain the list of all included studies, the subgroup of datasets analysed in this review, and the excluded studies with reasons, respectively. The remaining included studies are considered in other articles from the project [ 63 ]. The flow of our study is presented in Figure 1 .…”

“…Please refer to the protocol [ 62 ] and our other papers [ 63 ] from the project on AAER from CV signals with AI methods for additional analyses.…”

“…We considered any type of publication to be eligible in which CV signals and AI methods were used for AAER. The primary focus of our whole project [ 63 ] was the performance of these computer programs (e.g., specificity, sensitivity, accuracy). For this focused systematic review, we imposed additional inclusion criteria, namely public availability of the data.…”

“…The quantitative summary with sensitivity, heterogeneity, and subgroup analysis of all papers is not the purpose of this focused review. For more details, please refer to the protocol [ 62 ] and other papers from the project [ 63 ].…”

“…This paper is a part of the project on a systematic review of studies focused on AAER from CV signals with AI methods. For more details, see the protocol [ 62 ] and our previous conference paper [ 63 ].…”