Player Modeling via Multi-Armed Bandits

Abstract

This paper focuses on building personalized player models solely from player behavior in the context of adaptive games. We present two main contributions: The first is a novel approach to player modeling based on multi-armed bandits (MABs). This approach addresses, at the same time and in a principled way, both the problem of collecting data to model the characteristics of interest for the current player and the problem of adapting the interactive experience based on this model. Second, we present an approach to evaluating and fine-tuning these algorithms prior to generating data in a user study. This is an important problem, because conducting user studies is an expensive and labor-intensive process; therefore, an ability to evaluate the algorithms beforehand can save a significant amount of resources. We evaluate our approach in the context of modeling players' social comparison orientation (SCO) and present empirical results from both simulations and real players. CCS CONCEPTS • Computing methodologies → Artificial intelligence; • Humancentered computing → User models.

Regression Oracles and Exploration Strategies for Short-Horizon Multi-Armed Bandits

“…This paper focused on the problem of short-horizon multiarmed bandits, where we can only expect to have a small number of iterations to interact with the environment. These short-horizon problems are common in real-world situations, such as when using bandits to interact with human players in games (for example, for player modeling [6]), but they have been understudied in the literature. We presented three key ideas: regression oracles, a comparison of different exploration strategies with forced exploration in the short horizon setting, and a new variant of the UCB1 strategy called UCBT.…”

“…We would also like to analyze the theoretical properties of both UCBT and the inclusion of a regression oracle ingreedy strategies. Finally, in our current work, we have been deploying these ideas for player modeling in the context of exergames [6].…”

“…For example, consider the problem of player modeling [3]- [5], where software systems aim to model or classify players to provide them with individualized experiences. In our previous work [6], we demonstrated that MAB strategies can be useful in automatically determining the best options to present a user, based on that user's traits, as the MAB strategy observes the user's reactions to the different options. However, if MAB strategies are to be useful for player modeling, they need to adapt to users quickly within a small number of iterations.…”

The Personalization Paradox: the Conflict between Accurate User Models and Personalized Adaptive Systems

“…In our recent work on personalization in the context of physical activity (PA) [12], we designed an adaptive system based on social comparison [2] to motivate users to perform PA. Specifically, we designed a web-based platform in which users can compare themselves, including their daily steps with other users' PA-related profiles.…”

“…The user's steps are captured by Fitbit and synced automatically with our platform. We use the AI technique of multi-armed bandits to model individual users' social comparison preferences and adapt the comparison targets shown to them [12,13]. For example, if the user model predicts that a particular user tends to prefer upward comparisons, the system will show more profiles with a larger number of daily steps.…”