Improving Challenge/Skill Ratio in a Multimodal Interface by Simultaneously Adapting Game Difficulty and Haptic Assistance through Psychophysiological and Performance Feedback

Rodriguez–Guerrero, Carlos; Knaepen, Kristel; Fraile, Juan-Carlos; Pérez-Turiel, Javier; Gonzalez-de-Garibay, Valentin; Lefeber, Dirk

doi:10.3389/fnins.2017.00242

Cited by 19 publications

(15 citation statements)

References 61 publications

(121 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, affective games have the potential to achieve more effective adaptation than “classic” games (Ng et al, 2012) and consequently result in higher user engagement, immersion and enjoyment (Nagle et al, 2015; McCrea et al, 2016; Denisova and Cairns, 2018). Such improvements would be useful not only for entertainment, but also for serious game applications such as education (Ip et al, 2016), motor rehabilitation (Koenig et al, 2011; Rodriguez-Guerrero et al, 2017), and autism intervention (Zhang et al, 2017b).…”

Section: Introductionmentioning

confidence: 99%

“…In affective games, the most commonly used measures from the central nervous system are the electroencephalogram (EEG) (Ma et al, 2015), which records the electrical activity of the brain, and functional near infrared spectroscopy (Girouard et al, 2010), which records the hemodynamic activity associated with neural behavior. Measurements from the peripheral nervous system are largely associated with autonomic activation and include the electrocardiogram (ECG) (Rodriguez-Guerrero et al, 2017), which monitors the electrical activity of the heart (specifically heart rate), galvanic skin response (GSR) (Nourbakhsh et al, 2017), which records the activity of the skin’s sweat glands, skin temperature (ST), respiration rate (Picard et al, 2001), and others. Physiological measures are quantitative and sensitive to different kinds of stimuli, but are often affected by noise (Larson and Taulu, 2018).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Classification of Multiple Psychological Dimensions in Computer Game Players Using Physiology, Performance, and Personality Characteristics

et al. 2019

View full text Add to dashboard Cite

Human psychological (cognitive and affective) dimensions can be assessed using several methods, such as physiological or performance measurements. To date, however, few studies have compared different data modalities with regard to their ability to enable accurate classification of different psychological dimensions. This study thus compares classification accuracies for four psychological dimensions and two subjective preferences about computer game difficulty using three data modalities: physiology, performance, and personality characteristics. Thirty participants played a computer game at nine difficulty configurations that were implemented via two difficulty parameters. In each configuration, seven physiological measurements and two performance variables were recorded. A short questionnaire was filled out to assess the perceived difficulty, enjoyment, valence, arousal, and the way the participant would like to modify the two difficulty parameters. Furthermore, participants’ personality characteristics were assessed using four questionnaires. All combinations of the three data modalities (physiology, performance, and personality) were used to classify six dimensions of the short questionnaire into either two, three or many classes using four classifier types: linear discriminant analysis, support vector machine (SVM), ensemble decision tree, and multiple linear regression. The classification accuracy varied widely between the different psychological dimensions; the highest accuracies for two-class and three-class classification were 97.6 and 84.1%, respectively. Normalized physiological measurements were the most informative data modality, though current game difficulty, personality and performance also contributed to classification accuracy; the best selected features are presented and discussed in the text. The SVM and multiple linear regression were the most accurate classifiers, with regression being more effective for normalized physiological data. In the future, we will further examine the effect of different classification approaches on user experience by detecting the user’s psychological state and adapting game difficulty in real-time. This will allow us to obtain a complete picture of the performance of affect-aware systems in both an offline (classification accuracy) and real-time (effect on user experience) fashion.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Classification of Multiple Psychological Dimensions in Computer Game Players Using Physiology, Performance, and Personality Characteristics

et al. 2019

View full text Add to dashboard Cite

show abstract

“…For instance, Koenig et al [37] and Novak et al [38] used inputs from ANS such as skin-conductance level, heart rate and skin temperature to estimate the engagement and arousal level of stroke patients while performing rehabilitation movements, and proposed their use in developing a closedloop auto-adaptive rehabilitation program. Such systems have been shown to promote the engagement level and participation, and improve the overall user experience [9,10,39], which can increase the rate and amount of recovery from stroke disabilities [8,22,24]. However, stroke patients often show long-lasting abnormalities in ANS, which can alter the functioning of ANS-based auto-adaptive rehabilitation programs [40].…”

Section: Discussionmentioning

confidence: 99%

The influence of psychological and cognitive states on error-related negativity evoked during post-stroke rehabilitation movements

2021

View full text Add to dashboard Cite

Background Recently, error-related negativity (ERN) signals are proposed to develop an assist-as-needed robotic stroke rehabilitation program. Stroke patients’ state-of-mind, such as motivation to participate and active involvement in the rehabilitation program, affects their rate of recovery from motor disability. If the characteristics of the robotic stroke rehabilitation program can be altered based on the state-of-mind of the patients, such that the patients remain engaged in the program, the rate of recovery from their motor disability can be improved. However, before that, it is imperative to understand how the states-of-mind of a participant affect their ERN signal. Methods This study aimed to determine the association between the ERN signal and the psychological and cognitive states of the participants. Experiments were conducted on stroke patients, which involved performing a physical rehabilitation exercise and a questionnaire to measure participants' subjective experience on four factors: motivation in participating in the experiment, perceived effort, perceived pressure, awareness of uncompleted exercise trials while performing the rehabilitation exercise. Statistical correlation analysis, EEG time-series and topographical analysis were used to assess the association between the ERN signals and the psychological and cognitive states of the participants. Results A strong correlation between the amplitude of the ERN signal and the psychological and cognitive states of the participants was observed, which indicate the possibility of estimating the said states using the amplitudes of the novel ERN signal. Conclusions The findings pave the way for the development of an ERN based dynamically adaptive assist-as-needed robotic stroke rehabilitation program of which characteristics can be altered to keep the participants’ motivation, effort, engagement in the rehabilitation program high. In future, the single-trial prediction ability of the novel ERN signals to predict the state-of-mind of stroke patients will be evaluated.

show abstract

“…Physiological measures can unobtrusively quantify psychological states by measuring the physiological responses to such states. They include the electrocardiogram (ECG) ( Jung et al, 2014 ), which records HR ( Collet et al, 2009 ), galvanic skin response (GSR), which records the activity level of the skin’s sweat glands ( Bongiorno et al, 2017 ; Rodriguez-Guerrero et al, 2017 ), respiration rate (RR) ( Healey and Picard, 2005 ), skin temperature (ST) ( Kajiwara, 2014 ), eye gaze ( Fletcher and Zelinsky, 2009 ), blink frequency ( He et al, 2017 ), electroencephalography (EEG) ( Mühl et al, 2014 ; Mu et al, 2017 ), and others. They are quantitative and can be recorded in a real-time manner without the user’s active involvement, but are often affected by noise and difficult to interpret ( Healey and Picard, 2005 ).…”

Section: Introductionmentioning

confidence: 99%

Identifying the Causes of Drivers’ Hazardous States Using Driver Characteristics, Vehicle Kinematics, and Physiological Measurements

et al. 2018

View full text Add to dashboard Cite

Drivers’ hazardous physical and mental states (e.g., distraction, fatigue, stress, and high workload) have a major effect on driving performance and strongly contribute to 25–50% of all traffic accidents. They are caused by numerous factors, such as cell phone use or lack of sleep. However, while significant research has been done on detecting hazardous states, most studies have not tried to identify the causes of the hazardous states. Such information would be very useful, as it would allow intelligent vehicles to better respond to a detected hazardous state. Thus, this study examined whether the cause of a driver’s hazardous state can be automatically identified using a combination of driver characteristics, vehicle kinematics, and physiological measures. Twenty-one healthy participants took part in four 45-min sessions of simulated driving, of which they were mildly sleep-deprived for two sessions. Within each session, there were eight different scenarios with different weather (sunny or snowy), traffic density and cell phone usage (with or without cell phone). During each scenario, four physiological (respiration, electrocardiogram, skin conductance, and body temperature) and eight vehicle kinematics measures were monitored. Additionally, three self-reported driver characteristics were obtained: personality, stress level, and mood. Three feature sets were formed based on driver characteristics, vehicle kinematics, and physiological signals. All possible combinations of the three feature sets were used to classify sleep deprivation (drowsy vs. alert), traffic density (low vs. high), cell phone use, and weather conditions (foggy/snowy vs. sunny) with highest accuracies of 98.8%, 91.4%, 82.3%, and 71.5%, respectively. Vehicle kinematics were most useful for classification of weather and traffic density while physiology and driver characteristics were useful for classification of sleep deprivation and cell phone use. Furthermore, a second classification scheme was tested that also incorporates information about whether or not other causes of hazardous states are present, though this did not result in higher classification accuracy. In the future, these classifiers could be used to identify both the presence and cause of a driver’s hazardous state, which could serve as the basis for more intelligent intervention systems.

show abstract

Improving Challenge/Skill Ratio in a Multimodal Interface by Simultaneously Adapting Game Difficulty and Haptic Assistance through Psychophysiological and Performance Feedback

Cited by 19 publications

References 61 publications

Classification of Multiple Psychological Dimensions in Computer Game Players Using Physiology, Performance, and Personality Characteristics

Classification of Multiple Psychological Dimensions in Computer Game Players Using Physiology, Performance, and Personality Characteristics

The influence of psychological and cognitive states on error-related negativity evoked during post-stroke rehabilitation movements

Identifying the Causes of Drivers’ Hazardous States Using Driver Characteristics, Vehicle Kinematics, and Physiological Measurements

Contact Info

Product

Resources

About