Rising to the Challenge: An Emotion-Driven Approach Toward Adaptive Serious Games

Schrader, Claudia; Brich, Julia; Frommel, Julian; Riemer, Valentin; Rogers, Katja

doi:10.1007/978-3-319-51645-5_1

Cited by 23 publications

(17 citation statements)

References 89 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The analysis of facial behavior commonly relies on data obtained from physical sensors, for example, electromyography (EMG), or from the application of visual methods to assess the face, for example, feature extraction via computer vision [20]. The approach based on EMG data uses physical sensors attached to subjects to measure electrical activity of facial muscles, such as the zygomaticus, the orbicularis oculi, and the corrugator supercilii muscles (Figure 1), associated with smiling, eyelids control, and frowning, respectively.…”

Section: Related Workmentioning

confidence: 99%

Automated Analysis of Facial Cues from Videos as a Potential Method for Differentiating Stress and Boredom of Players in Games

Bevilacqua

Engström

Backlund

2018

International Journal of Computer Games Technology

View full text Add to dashboard Cite

Facial analysis is a promising approach to detect emotions of players unobtrusively; however approaches are commonly evaluated in contexts not related to games or facial cues are derived from models not designed for analysis of emotions during interactions with games. We present a method for automated analysis of facial cues from videos as a potential tool for detecting stress and boredom of players behaving naturally while playing games. Computer vision is used to automatically and unobtrusively extract 7 facial features aimed at detecting the activity of a set of facial muscles. Features are mainly based on the Euclidean distance of facial landmarks and do not rely on predefined facial expressions, training of a model, or the use of facial standards. An empirical evaluation was conducted on video recordings of an experiment involving games as emotion elicitation sources. Results show statistically significant differences in the values of facial features during boring and stressful periods of gameplay for 5 of the 7 features. We believe our approach is more user-tailored, convenient, and better suited for contexts involving games.

show abstract

Section: Related Workmentioning

confidence: 99%

Automated Analysis of Facial Cues from Videos as a Potential Method for Differentiating Stress and Boredom of Players in Games

Bevilacqua

Engström

Backlund

2018

International Journal of Computer Games Technology

View full text Add to dashboard Cite

show abstract

“…While "fun" is the locus of attention in entertainment games and has been investigated in learning games as well (Nebel et al, 2017c), educational games serve additional purposes, as they need to convey learning content appropriately to learners. According to Schrader et al (2017) adaptivity in educational games can be defined as "a player-centred approach by adjusting game's mechanics and representational modes to suit game's responsiveness to player characteristics with the purpose of improving in-game behavior, learning processes, and performance" (p. 5). Hence, finding the right balance between the learner's skills and the challenge levels of the games is a critical issue, especially as the perceived difficulty and inferred feedback after facing a task could influence learning outcomes (e.g., Nebel et al, 2017b).…”

Section: Introductionmentioning

confidence: 99%

A Systematic Literature Review of Analytics for Adaptivity Within Educational Video Games

Ninaus

Nebel

2021

Front. Educ.

View full text Add to dashboard Cite

Research has shown that serious games, digital game-based learning, and educational video games can be powerful learning instruments. However, experimental and meta-research have revealed that several moderators and variables influence the resulting learning outcomes. Advances in the areas of learning and game analytics potentially allow for controlling and improving the underlying learning processes of games by adapting their mechanics to the individual needs of the learner, to properties of the learning material, and/or to environmental factors. However, the field is young and no clear-cut guidelines are yet available. To shed more light on this topic and to identify common ground for further research, we conducted a systematic and pre-registered analysis of the literature. Particular attention was paid to different modes of adaptivity, different adaptive mechanisms in various learning domains and populations, differing theoretical frameworks, research methods, and measured concepts, as well as divergent underlying measures and analytics. Only 10 relevant papers were identified through the systematic literature search, which confirms that the field is still in its very early phases. The studies on which these papers were based, however, show promise in terms of the efficacy of adaptive educational games. Moreover, we identified an increased interest in the field of adaptive educational games and in the use of analytics. Nevertheless, we also identified a clear lack of common theoretical foundations as well as the application of rather heterogenous methods for investigating the effects of adaptivity. Most problematic was the lack of sufficient information (e.g., descriptions of used games, adaptive mechanisms), which often made it difficult to draw clear conclusions. Future studies should therefore focus on strong theory building and adhere to reporting standards across disciplines. Researchers from different disciplines must act in concert to advance the current state of the field in order to maximize its potential.

show abstract

“…Contrary to the obtrusiveness of measuring signals using physical sensors, the extraction of signals using computer vision is presented as a viable, non-obtrusive alternative [65]. Remote photoplethysmography (rPPG) [66] is commonly used to estimate the HR of a person from the analysis of a video [67].…”

Section: Background and Related Workmentioning

confidence: 99%

Game-Calibrated and User-Tailored Remote Detection of Stress and Boredom in Games

Bevilacqua

Engström

Backlund

2019

Sensors

View full text Add to dashboard Cite

Emotion detection based on computer vision and remote extraction of user signals commonly rely on stimuli where users have a passive role with limited possibilities for interaction or emotional involvement, e.g., images and videos. Predictive models are also trained on a group level, which potentially excludes or dilutes key individualities of users. We present a non-obtrusive, multifactorial, user-tailored emotion detection method based on remotely estimated psychophysiological signals. A neural network learns the emotional profile of a user during the interaction with calibration games, a novel game-based emotion elicitation material designed to induce emotions while accounting for particularities of individuals. We evaluate our method in two experiments ( n = 20 and n = 62 ) with mean classification accuracy of 61.6%, which is statistically significantly better than chance-level classification. Our approach and its evaluation present unique circumstances: our model is trained on one dataset (calibration games) and tested on another (evaluation game), while preserving the natural behavior of subjects and using remote acquisition of signals. Results of this study suggest our method is feasible and an initiative to move away from questionnaires and physical sensors into a non-obtrusive, remote-based solution for detecting emotions in a context involving more naturalistic user behavior and games.

show abstract

Rising to the Challenge: An Emotion-Driven Approach Toward Adaptive Serious Games

Cited by 23 publications

References 89 publications

Automated Analysis of Facial Cues from Videos as a Potential Method for Differentiating Stress and Boredom of Players in Games

Automated Analysis of Facial Cues from Videos as a Potential Method for Differentiating Stress and Boredom of Players in Games

A Systematic Literature Review of Analytics for Adaptivity Within Educational Video Games

Game-Calibrated and User-Tailored Remote Detection of Stress and Boredom in Games

Contact Info

Product

Resources

About