A Survey of Video Game Testing

Politowski, Cristiano; Petrillo, Fábio; Guéhéneuc, Yann‐Gaël

doi:10.48550/arxiv.2103.06431

Cited by 5 publications

(6 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Automating software testing is a long standing problem in software development. The problem is however notably different in video games when compared to other software [15,18,13]. Video games have attributes that most other software do not, they tend to present a narrative, have a rich presentation layer, and are highly interactive.…”

Section: Bug Identificationmentioning

confidence: 99%

Learning to Identify Perceptual Bugs in 3D Video Games

Wilkins¹,

Stathis²

2022

Preprint

View full text Add to dashboard Cite

Automated Bug Detection (ABD) in video games is composed of two distinct but complementary problems: automated game exploration and bug identification. Automated game exploration has received much recent attention, spurred on by developments in fields such as reinforcement learning. The complementary problem of identifying the bugs present in a player's experience has for the most part relied on the manual specification of rules. Although it is widely recognised that many bugs of interest cannot be identified with such methods, little progress has been made in this direction. In this work we show that it is possible to identify a range of perceptual bugs using learning-based methods by making use of only the rendered game screen as seen by the player. To support our work, we have developed World of Bugs (WOB), an open platform for testing ABD methods in 3D game environments.

show abstract

Section: Bug Identificationmentioning

confidence: 99%

Learning to Identify Perceptual Bugs in 3D Video Games

Wilkins¹,

Stathis²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Three recent studies [33,40,46] suggest that finding performance issues in video games is a relevant problem, according to both game developers [40,46] and players [33]. Nevertheless, to the best of our knowledge, no previous work introduced automated approaches for load testing video games.…”

Section: Threats To Validitymentioning

confidence: 99%

“…In a recent survey, Politowski et al [40] reported that for two out of the five games they considered (i.e., League of Legends by Riot and Sea of Thieves by Rare) developers partially automated game performance checks (e.g., frame-rate). Similarly, Naughty Dog used specialized profiling tools 1 for finding which parts of a given scene caused a drop in the number of frames per second (FPS) in The Last of Us.…”

Section: Introductionmentioning

confidence: 99%

Using Reinforcement Learning for Load Testing of Video Games

Tufano¹,

Scalabrino²,

Pascarella³

et al. 2022

Preprint

View full text Add to dashboard Cite

Different from what happens for most types of software systems, testing video games has largely remained a manual activity performed by human testers. This is mostly due to the continuous and intelligent user interaction video games require. Recently, reinforcement learning (RL) has been exploited to partially automate functional testing. RL enables training smart agents that can even achieve super-human performance in playing games, thus being suitable to explore them looking for bugs. We investigate the possibility of using RL for load testing video games. Indeed, the goal of game testing is not only to identify functional bugs, but also to examine the game's performance, such as its ability to avoid lags and keep a minimum number of frames per second (FPS) when highdemanding 3D scenes are shown on screen. We define a methodology employing RL to train an agent able to play the game as a human while also trying to identify areas of the game resulting in a drop of FPS. We demonstrate the feasibility of our approach on three games. Two of them are used as proof-of-concept, by injecting artificial performance bugs. The third one is an open-source 3D game that we load test using the trained agent showing its potential to identify areas of the game resulting in lower FPS.

show abstract

“…There are many unique challenges when applying general software engineering practices in video game development [32,37,40,43,48], including challenges in game testing. Manual testing is a widely accepted approach to game testing [39,42,52], however this manual process is slow and error-prone, and most importantly, expensive. On the other hand, it is challenging to automate game testing [31,39,47] due to the unpredictable outputs of video games.…”

Section: Introductionmentioning

confidence: 99%

CLIP meets GamePhysics: Towards bug identification in gameplay videos using zero-shot transfer learning

Taesiri¹,

Macklon²,

Bezemer³

2022

Preprint

View full text Add to dashboard Cite

Gameplay videos contain rich information about how players interact with the game and how the game responds. Sharing gameplay videos on social media platforms, such as Reddit, has become a common practice for many players. Often, players will share gameplay videos that showcase video game bugs. Such gameplay videos are software artifacts that can be utilized for game testing, as they provide insight for bug analysis. Although large repositories of gameplay videos exist, parsing and mining them in an effective and structured fashion has still remained a big challenge. In this paper, we propose a search method that accepts any English text query as input to retrieve relevant videos from large repositories of gameplay videos. Our approach does not rely on any external information (such as video metadata); it works solely based on the content of the video. By leveraging the zero-shot transfer capabilities of the Contrastive Language-Image Pre-Training (CLIP) model, our approach does not require any data labeling or training. To evaluate our approach, we present the GamePhysics dataset consisting of 26,954 videos from 1,873 games, that were collected from the GamePhysics section on the Reddit website. Our approach shows promising results in our extensive analysis of simple queries, compound queries, and bug queries, indicating that our approach is useful for object and event detection in gameplay videos. An example application of our approach is as a gameplay video search engine to aid in reproducing video game bugs. Please visit the following link for the code and the data: https://asgaardlab.github.io/CLIPxGamePhysics/ CCS CONCEPTS• Software and its engineering → Software testing and debugging.

show abstract

A Survey of Video Game Testing

Cited by 5 publications

References 28 publications

Learning to Identify Perceptual Bugs in 3D Video Games

Learning to Identify Perceptual Bugs in 3D Video Games

Using Reinforcement Learning for Load Testing of Video Games

CLIP meets GamePhysics: Towards bug identification in gameplay videos using zero-shot transfer learning

Contact Info

Product

Resources

About