Explainable Multimodal Machine Learning for Engagement Analysis by Continuous Performance Test

Rahman, Muhammad Arifur; Brown, David J.; Shopland, Nicholas; Burton, Audrey; Mahmud, Mufti

doi:10.1007/978-3-031-05039-8_28

Cited by 38 publications

(8 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ahuja and Banga [ 57 ] created another dataset where they classified mental stress in 206 students. They used linear regression (LR), support vector machine (SVM), Naïve Bayes (NB) and random forest (RF) ML classification algorithms [ 9 , 30 , 38 , 41 , 49 , 51 , 58 – 60 ] to determine mental stress. Using SVM and tenfold cross-validation, they claimed an 85.71% accuracy.…”

Section: Related Workmentioning

confidence: 99%

“…However, a reliable and automated system is needed to accomplish this task. Given that artificial intelligence (AI) and machine learning (ML) have been playing significant roles in the methodological developments for diverse problem domains, including computational biology [ 9 , 10 ], cyber security [ 11 – 14 ], disease detection [ 15 – 21 ] and management [ 22 – 27 ], elderly care [ 28 , 29 ], epidemiological study [ 30 ], fighting pandemic [ 31 – 37 ], healthcare [ 38 – 42 ], healthcare service delivery [ 43 – 45 ], natural language processing [ 46 – 50 ], social inclusion [ 51 – 53 ] and many more, the AI and ML-based methods can be employed to do this task. Hence, here we have explored a series of ML models with publicly available data sets (using electroencephalogram and heart rate variability) to predict arousal states.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Enhancing biofeedback-driven self-guided virtual reality exposure therapy through arousal detection from multimodal data using machine learning

et al. 2023

Self Cite

View full text Add to dashboard Cite

Virtual reality exposure therapy (VRET) is a novel intervention technique that allows individuals to experience anxiety-evoking stimuli in a safe environment, recognise specific triggers and gradually increase their exposure to perceived threats. Public-speaking anxiety (PSA) is a prevalent form of social anxiety, characterised by stressful arousal and anxiety generated when presenting to an audience. In self-guided VRET, participants can gradually increase their tolerance to exposure and reduce anxiety-induced arousal and PSA over time. However, creating such a VR environment and determining physiological indices of anxiety-induced arousal or distress is an open challenge. Environment modelling, character creation and animation, psychological state determination and the use of machine learning (ML) models for anxiety or stress detection are equally important, and multi-disciplinary expertise is required. In this work, we have explored a series of ML models with publicly available data sets (using electroencephalogram and heart rate variability) to predict arousal states. If we can detect anxiety-induced arousal, we can trigger calming activities to allow individuals to cope with and overcome distress. Here, we discuss the means of effective selection of ML models and parameters in arousal detection. We propose a pipeline to overcome the model selection problem with different parameter settings in the context of virtual reality exposure therapy. This pipeline can be extended to other domains of interest where arousal detection is crucial. Finally, we have implemented a biofeedback framework for VRET where we successfully provided feedback as a form of heart rate and brain laterality index from our acquired multimodal data for psychological intervention to overcome anxiety.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Enhancing biofeedback-driven self-guided virtual reality exposure therapy through arousal detection from multimodal data using machine learning

et al. 2023

Self Cite

View full text Add to dashboard Cite

show abstract

“…Diagnosis of DR can be performed through either manual examination by an ophthalmologist or by utilising an automated system. With the advancements in Artificial Intelligence (AI) techniques, automated system development has been facilitated in many application areas including anomaly detection [ 3 ], brain signal analysis [ 4 ], neurodevelopmental disorder assessment and classification focusing on autism [ 5 , 6 , 7 ], neurological disorder detection and management [ 8 ], supporting the detection and management of the COVID-19 pandemic [ 9 ], cyber security and trust management [ 10 , 11 , 12 , 13 ], various disease diagnosis [ 14 , 15 , 16 , 17 ], smart healthcare service delivery [ 18 , 19 ], text and social media mining [ 20 , 21 ], understanding student engagement [ 22 , 23 ], etc. As can be seen in the literature, automated systems for early disease detection have been a major area of development.…”

Section: Introductionmentioning

confidence: 99%

Classification of Diabetic Retinopathy Disease Levels by Extracting Spectral Features Using Wavelet CNN

Sundar,

Subramanian,

Mahmud

2024

Diagnostics

Self Cite

View full text Add to dashboard Cite

Diabetic retinopathy (DR) arises from blood vessel damage and is a leading cause of blindness on a global scale. Clinical professionals rely on examining fundus images to diagnose the disease, but this process is frequently prone to errors and is tedious. The usage of computer-assisted techniques offers assistance to clinicians in detecting the severity levels of the disease. Experiments involving automated diagnosis employing convolutional neural networks (CNNs) have produced impressive outcomes in medical imaging. At the same time, retinal image grading for detecting DR severity levels has predominantly focused on spatial features. More spectral features must be explored for a more efficient performance of this task. Analysing spectral features plays a vital role in various tasks, including identifying specific objects or materials, anomaly detection, and differentiation between different classes or categories within an image. In this context, a model incorporating Wavelet CNN and Support Vector Machine has been introduced and assessed to classify clinically significant grades of DR from retinal fundus images. The experiments were conducted on the EyePACS dataset and the performance of the proposed model was evaluated on the following metrics: precision, recall, F1-score, accuracy, and AUC score. The results obtained demonstrate better performance compared to other state-of-the-art techniques.

show abstract

“…This might speed up the development process, lower expenses, and use fewer resources.In research and development connected to usability, cognitive architectures and cognitive models have been shown to have potential.However, because there aren't many tools available to support cognitive modelling, this approach isn't frequently used in usability research and development. The examination of data from the cognitive model is the main focus of this study [3]. BCI is gaining popularity as a cutting-edge solution for facilitating communication between IoT products and people.…”

Section: Introductionmentioning

confidence: 99%

Body Sensor Cloud Network Based Data Classification By Machine Learning Techniques In Cognitive Human Computer Interaction

D¹,

S.S

Shukla

et al. 2023

Preprint

View full text Add to dashboard Cite

Recent developments in cognitive technical systems (CTS), which offer organic and effective operating principles, reveal a development in human-computer interaction (HCI). A CTS must rely on data from several sensors, which must then be processed and merged by fusion algorithms, to do this. To put the observations made into the proper context, additional knowledge sources must also be integrated. This research propose novel technique in cognitive human computer interaction based body sensor data analytics using machine learning technique. here the body sensor based monitoring data has been collected and transmitted by cloud networks for cognitive human computer interaction. then this data has been processed and trained using Boltzmann perceptron basis encoder neural network. Various body sensor-based monitored datasets are subjected to experimental analysis for accuracy, precision, recall, F-1 score, RMSE, normalised square error (NSE), and mean average precision. Proposed technique obtained 93% accuracy, 79% precision, 72% of recall, 64% f-1 score, 51% of RMSE, 56% NSE and 48% MAP.

show abstract

Explainable Multimodal Machine Learning for Engagement Analysis by Continuous Performance Test

Cited by 38 publications

References 41 publications

Enhancing biofeedback-driven self-guided virtual reality exposure therapy through arousal detection from multimodal data using machine learning

Enhancing biofeedback-driven self-guided virtual reality exposure therapy through arousal detection from multimodal data using machine learning

Classification of Diabetic Retinopathy Disease Levels by Extracting Spectral Features Using Wavelet CNN

Body Sensor Cloud Network Based Data Classification By Machine Learning Techniques In Cognitive Human Computer Interaction

Contact Info

Product

Resources

About