Towards Deep Player Behavior Models in MMORPGs

Abstract

Due to a steady increase in popularity, player demands for video game content are growing to an extent at which consistency and novelty in challenges are hard to attain. Problems in balancing and error-coping accumulate. To tackle these challenges, we introduce deep player behavior models, applying machine learning techniques to individual, atomic decisionmaking strategies. We discuss their potential application in personalized challenges, autonomous game testing, human agent substitution, and online crime detection. Results from a pilot study that was carried out with the massively multiplayer online role-playing game Lineage II depict a benchmark between hidden markov models, decision trees, and deep learning. Data analysis and individual reports indicate that deep learning can be employed to provide adequate models of individual player behavior with high accuracy for predicting skill-use and a high correlation in recreating strategies from previously recorded data. CCS Concepts•Information systems → Massively multiplayer online games; •Human-centered computing → User models; •Computing methodologies → Machine learning approaches;

Bot or not? User Perceptions of Player Substitution with Deep Player Behavior Models

“…In previous work, we were successful in showing that player modeling agents yield significantly higher motivation potential than heuristic opponents [36]. In addition, we contrasted different machine learning techniques in a player modeling study of the MMORPG Lineage 2 [33,35], showing that deep learning offers the highest individual prediction accuracies with the ability to reproduce playing sessions that closely resemble the original behavior, as well as offering the potential to differentiate between players. Based on this, we embedded DPBM into a long-term DDA evaluation about competing against agents of own behavior on a daily basis in the MMORPG AION [37], in which DPBM opponents were perceived to be significantly more engaging than traditional DDA opponents adjusted by heuristic parameter tuning.…”

“…Based on insights about expressive data and suitable modeling techniques from our earlier work [35,36], we recorded all crucial player action decisions (attacking with -or switching to -a specific element and jumping) together with situational data from the current game state. After every level and for each player, the recorded behavioral data from all preceding levels was fed into a dedicated 24x10x10x9 feed-forward multi-layer perceptron with backpropagation and a logistic sigmoid activation function (cf.…”

“…Figure 2). The network was initialized randomly and trained in a background thread over 1000 epochs, based on previous findings [35,36] and benchmarks prior to the study that indicated diminishing returns beyond these parameters. When a DPBM bot substituted a player, it applied the trained model generatively to retrieve a set of action probabilities based on the given state description in real-time.…”

“…Based on the insights of previous work [35,36], we chose to endow the heuristic bots with random decision making CHI 2020, April 25-30, 2020, Honolulu, HI, USA between elements, since it yields a balanced performance level, analogous to random decision making in rock-paperscissors. Thus, contrary to human and DPBM opponents, it was impossible for other players to predict this behavior.…”

MarioMix

“…For instance, creating bots with varied playstyles can be useful for game designers in early development stages, such as playtesting using bots [5,25]. Unfortunately, such bot behaviors have not been widely adopted in the game industry.…”