Fusing Visual and Behavioral Cues for Modeling User Experience in Games

Abstract

Abstract-Estimating affective and cognitive states in conditions of rich human-computer interaction, such as in games, is a field of growing academic and commercial interest. Entertainment and serious games can benefit from recent advances in the field as, having access to predictors of the current state of the player (or learner) can provide useful information for feeding adaptation mechanisms that aim to maximize engagement or learning effects. In this paper, we introduce a large data corpus derived from 58 participants that play the popular Super Mario Bros platform game and attempt to create accurate models of player experience for this game genre. Within the view of the current research, features extracted both from player gameplay behavior and game levels, and player visual characteristics have been used as potential indicators of reported affect expressed as pairwise preferences between different game sessions. Using neuroevolutionary preference learning and automatic feature selection, highly accurate models of reported engagement, frustration, and challenge are constructed (model accuracies reach 91%, 92% and 88% for engagement, frustration and challenge, respectively). As a step further, the derived player experience models can be used to personalize the game level to desired levels of engagement, frustration and challenge, as game content is mapped to player experience through the behavioral and expressivity patterns of each player.

The platformer experience dataset

“…game level content) representation, self-reported experience via rating and ranking questionnaire schemes and player demographics. The experimental results already obtained on this dataset [9], [10], and briefly presented here, indicate that the visually extracted information and the gameplay features recorded are appropriate for multimodal player modeling and for testing player experience models using different sets of features and data representations.…”

“…While the majority of player experience modeling studies focus on the relationship between in-game behavioral data and experience annotations (e.g. see [15], [16], [17], [18], [9] among others), the literature is sparse investigating the interplay between visual and ingame behavior of users and its association to self-reported experience and game content.…”

“…Regarding player experience, Shaker et al [9] conducted a set of experiments on the PE dataset to test the hypothesis that the use of multimodal channels of information about player behavior would result in more accurate models of player experience than the ones constructed from a single input modality. A number of player experience models were constructed from four different subsets of the information extracted from the rich information provided in the dataset.…”

Constructive generation methods for dungeons and levels

“…The game was created to replace IMB which is the platform extensively used in research [18,28,32,9,2] and the software for the Mario AI Championship [26,24,8]. Since the level generator for InfiTux is the same as the one used for IMB, the game features infinite variations of levels by the use of a random seed.…”

The experience-driven perspective

“…The models built on this user input type rely on detailed attributes from the player's behaviour which are extracted from player behavioural responses during the interaction with game content stimuli. Such attributes, also named game metrics, are statistical spatio-temporal features of game interaction [6] which are usually mapped to levels of cognitive states such as attention, challenge and engagement [23]. In general, both generic measures-such as the level of player performance and the time spent on a task-as well as game-specific measures-such as the items picked and used-are relevant for the gameplay-based PEM.…”