Multiplayer Modeling via Multi-Armed Bandits

Gray, Robert C.; Zhu, Jichen; Ontañón, Santiago

doi:10.1109/cog52621.2021.9618892

Cited by 5 publications

(4 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…EM agents construct a model using the AI for a player and, in the case of software-based interventions like serious games, leverage this understanding of the player to tailor the gaming experience for the player toward its particular goals, such as health outcomes (Fujiki et al 2008) or learning objectives (Valls-Vargas et al 2015). The EM agent is able to perform this tailoring through the use of EM levers (Gray, Zhu, and Ontañón 2021), or elements within the game environment that provide the agent with opportunities to affect the game state. Essential to our own study scenario and method (Section 3), other studies have found success in using additional characters in the game environment as EM levers (Feltz et al 2014;Samendinger et al 2017).…”

Section: Player Modeling and Experience Managementmentioning

confidence: 99%

“…We aim to create a simulation in which we can evaluate the potential efficacy of the Shapley Bandit prior to conducting human user studies. To do so, we employ virtual players provisioned with models that adjust their behavior according to their exposure to different social comparisons (discussed in Section 3) and described in further detail in our previous simulation study (Gray, Zhu, and Ontañón 2021). In short, virtual players will either be motivated or demotivated (in terms of daily step activity) based on their exposure to upward or downward comparisons and depending on their (randomized) internal preferences for such comparisons.…”

Section: Simulation Environmentmentioning

confidence: 99%

See 1 more Smart Citation

Distribution Fairness in Multiplayer AI Using Shapley Constraints

Gray,

Zhu,

Ontañón

2023

AIIDE

View full text Add to dashboard Cite

Experience management (EM) agents in multiplayer serious games face unique challenges and responsibilities regarding the fair treatment of players. One such challenge is the Greedy Bandit Problem that arises when using traditional Multi-Armed Bandits (MABs) as EM agents, which results in some players routinely prioritized while others may be ignored. We will show that this problem can be a cause of player non-adherence in a multiplayer serious game played by human users. To mitigate this effect, we propose a new bandit strategy, the Shapley Bandit, which enforces fairness constraints in its treatment of players based on the Shapley Value. We evaluate our approach via simulation with virtual players, finding that the Shapley Bandit can be effective in providing more uniform treatment of players while incurring only a slight cost in overall performance to a typical greedy approach. Our findings highlight the importance of fair treatment among players as a goal of multiplayer EM agents and discuss how addressing this issue may lead to more effective agent operation overall. The study contributes to the understanding of player modeling and EM in serious games and provides a promising approach for balancing fairness and engagement in multiplayer environments.

show abstract

Section: Player Modeling and Experience Managementmentioning

confidence: 99%

Section: Simulation Environmentmentioning

confidence: 99%

Distribution Fairness in Multiplayer AI Using Shapley Constraints

Gray,

Zhu,

Ontañón

2023

AIIDE

View full text Add to dashboard Cite

show abstract

“…Our approach works within the space of player modeling focused experience management as having an accurate and useful model of the player can allow for more intelligent ExpM actions (Sharma et al 2007(Sharma et al , 2010Yu and Riedl 2013). These have more recently been expanded into multiplayer environments (Gray, Zhu, and Ontañón 2021;Zhu and Ontañón 2019). In each of these works, player preferences are assumed to be static.…”

Section: Experience Management and Player Modelingmentioning

confidence: 99%

“…MABs have been used in player modeling before with the focus of adapting games to their players (Gray et al 2020;Gray, Zhu, and Ontañón 2021), but these are concerned with finding player models from scratch, rather than correcting one in live gameplay. As such while many of their goals are the same, such as finding a player model quickly, they have more freedom in how to modify the environment and can make more assumptions.…”

Section: Experience Management and Player Modelingmentioning

confidence: 99%

Using Multi-Armed Bandits to Dynamically Update Player Models in an Experience Managed Environment

Vinogradov

Harrison

2022

AIIDE

View full text Add to dashboard Cite

Players are often considered to be static in their preferred play styles, but this is often untrue. While in most games this is not an issue, in games where experience managers (ExpMs) control the experience, a shift in a player's preferences can lead to loss of engagement and churn. When an ExpM makes changes to the game world, the game world is now biased in favor of the current player model which will then influence how the ExpM will observe the player's actions, potentially leading to a biased and incorrect player model. In these situations, it is beneficial for the ExpM to recalculate the player model in an efficient manner. In this paper we show that we can use the techniques used to solve multi-armed bandits along with our own idea of distractions to minimize the time it takes to identify what a player's preferences are after they change, compensate for the bias of the game world, and to minimize the number of intrusive elements added to the game world. To evaluate these claims, we use a text-only interactive fiction environment specifically created to be experience managed and to exhibit bias. Our experiments show that multi-armed bandit algorithms can quickly recalculate a player model in response to shifts in a player's preferences compared to several baseline methods.

show abstract