Classification Accuracy from the Perspective of the User

Abstract

The accurate classification of psychophysiological data is an important determinant of the quality when interacting with a physiological computing system. Previous research has focused on classification accuracy of psychophysiological data in purely mathematical terms but little is known about how accuracy metrics relate to users' perceptions of accuracy during real-time interaction. A group of 14 participants watched a series of movie trailers and were asked to subjectively indicate their level of interest in a binary high/low fashion. Psychophysiological data (EEG, ECG and SCL) were used to create a binary classification of interest via a Support Vector Machine (SVM) algorithm. After a period of training, participants received real-time feedback from the classification algorithm and perceptions of accuracy were assessed. The purpose of the study was to compare mathematical classification accuracy with the perceived accuracy of the system as experienced by the users. Results indicated that perceived accuracy was subject to a number of psychological biases resulting from expectations, entrainment and development of trust. The F1 score was generally a significant predictor of perceived accuracy.

A Review of Psychophysiological Measures to Assess Cognitive States in Real-World Driving

“…Such studies have integrated data from more than one measure by conducting multi-modal analysis to extract the relevant features to capture the psychological phenomena at hand. A comparison of multiple classifiers to train & optimize machine learning algorithms can help determine the best fitting model to represent changes in cognitive states that can explain driving performance (Nadeau and Bengio, 2000; Fairclough et al, 2015; Balters and Steinert, 2017; Tran et al, 2017). Thus, utilizing multi-modal physiological signals, models could be trained to learn and predict motorists' sub-optimal cognitive states associated with unsafe-driving behavior.…”

Grand Challenges in Neurotechnology and System Neuroergonomics

“…When errors do occur, error recovery mechanisms must be seamlessly integrated into the interaction with the user. Similarly, we lack a strong understanding of how classification rates derived from machine learning algorithms regarding the internal state of the user correspond with those subjective types of self-assessment that informs perceived accuracy of the system (Fairclough et al, 2015 ; McCrea et al, 2016 ). Due to the high speed of data exchange between brain and machine, interactions with neurotechnology can occur implicitly and autonomously, i.e., functions can be activated without seeking confirmation from the user (Solovey et al, 2015 ; Serim and Jacucci, 2019 ).…”

Exploring Central-Peripheral Nervous System Interaction Through Multimodal Biosignals: A Systematic Review

“…Some of the most used stimuli are musical videos [18], [48], [63]- [65], visual stimuli [23], [40], [41], [45], [53], [54], [67], images [31], [33], [36], [46], [50], [51], [68], [69], Audio [29], [34], [37], [52], task based stimuli [24], [25], [27], [38], [39], [42]- [44], [48], [55]- [57], [58], [62], [70]- [75], film clips [76], [77], normal video clips [26], [62], [78]. In task-based stimuli either the subjects are instructed to do the mental task (mathematics-related problems, memorizing, computer-based gaming, and reading) or physical task (coldpressor test, rope skipping, surgical task, and fatigue exercise).…”