Intelligence at play: game-based assessment using a virtual-reality application

Simons, Alexander; Wohlgenannt, Isabell; Zelt, Sarah; Weinmann, Markus; Schneider, Johannes

doi:10.1007/s10055-023-00752-9

Cited by 4 publications

(3 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These games were developed for entertainment, but they are used for assessment under certain circumstances. An example is the virtual reality game Job Simulator [ 15 ].…”

Section: Introductionmentioning

confidence: 99%

Are serious games an alternative to traditional personality questionnaires? Initial analysis of a gamified assessment

Ramos-Villagrasa,

Fernández-del-Río,

Hermoso

et al. 2024

PLoS ONE

View full text Add to dashboard Cite

Personality questionnaires stand as crucial instruments in personnel selection but their limitations turn the interest towards alternatives like game-related assessments (GRAs). GRAs developed for goals other than fun are called serious games. Within them, gamified assessments are serious games that share similarities with traditional assessments (questionnaires, situational judgment tests, etc.) but they incorporate game elements like story, music, and game dynamics. This paper aims to contribute to the research on serious games as an alternative to traditional personality questionnaires by analyzing the characteristics of a gamified assessment called VASSIP. This gamified assessment, based on an existing measure of the Big Five personality traits, incorporates game elements such as storyfication, immersion, and non-evaluable gamified dynamics. The study performed included 98 university students (77.6% with job experience) as participants. They completed the original personality measure (BFI-2-S), the gamified evaluation of personality (VASSIP), a self-report measure of the main dimensions of job performance (task performance, contextual performance, and counterproductive work behaviors), and measures of applicant reactions to BFI-2-S and VASSIP. Results showed that the gamified assessment behaved similarly to the original personality measure in terms of reliability and participants’ scores, although the scores in Conscientiousness were substantially higher in VASSIP. Focusing on self-reports of the three dimensions of job performance, regression models showed that the gamified assessment could explain all of them. Regarding applicant reactions, the gamified assessment obtained higher scores in perceptions of comfort, predictive validity, and attractiveness, although the effect size was small except for the latter. Finally, all applicant reactions except for attractiveness were related to age and personality traits. In conclusion, gamified assessments have the potential to be an alternative to traditional personality questionnaires but VASSIP needs more research before its application in actual selection processes.

show abstract

“…These games were developed for entertainment, but they are used for assessment under certain circumstances. An example is the virtual reality game Job Simulator [ 15 ].…”

Section: Introductionmentioning

confidence: 99%

Are serious games an alternative to traditional personality questionnaires? Initial analysis of a gamified assessment

Ramos-Villagrasa,

Fernández-del-Río,

Hermoso

et al. 2024

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…However, some MOBAs (Multiplayer Online Battle Arena games) have also been analyzed (see Kokkinakis et al 2017). Recently, Simons et al (2023) examined the feasibility of a virtual reality video game about job performance to assess intelligence during personnel selection. Therefore, previous results showed that the two types of instruments (video games and tests) similarly sorted out individual differences among people.…”

Section: Introductionmentioning

confidence: 99%

Checking Different Video Game Mechanics to Assess Cognitive Abilities in Groups with and without Emotional Problems

Román,

Gutiérrez,

Ramos-Cejudo

et al. 2023

J. Intell.

View full text Add to dashboard Cite

This study assesses cognitive abilities through video games for entertainment (Blek, Edge, and Unpossible) that were programmed from scratch to record players’ behavior and the levels achieved in a sample without emotional problems and in one with emotional problems. The non-emotional-problem sample was recruited from three universities and two bachelor’s degree programs. The emotional-problem sample was recruited from two outpatient centers. The participants in the emotional-problem sample completed reduced versions of the ability tests and video games, as required by their emotional problems. Three subtests of the Differential Aptitude Test that assessed abstract reasoning, visuospatial reasoning, and perceptual speed were selected as ability tests. All participants were required to complete a mental health questionnaire (PROMIS) and a brief questionnaire on their gaming habits and previous experience with the video games used. The results that were obtained showed good convergent validity of the video games as measures of cognitive abilities, and they showed that the behavior of players in the sample without emotional problems while playing predicted the level achieved in the Blek and Unpossible game fragments, but this was only true for Unpossible in the emotional-problem sample; finally, shorter versions of the Blek and Edge game fragments can be used because they maintain their good psychometric properties.

show abstract

“…Consequently, any kind of construct-related validity or predictive validity study is needed. An example is the virtual reality game Job Simulator (Simons et al, 2023). Ramos-Villagrasa et al (2022) use this classification to clarify mixed results of prior research.…”

mentioning

confidence: 99%

Predictive Validity, Applicant Reactions, and Influence of Personal Characteristics of a Gamefully Designed Assessment

Ramos-Villagrasa,

Fernández-del-Río

2023

Journal of Work and Organizational Psychology

View full text Add to dashboard Cite

The rise of digital personnel selection procedures has led to a growing interest in new assessment methods (Andrés et al., 2023), such as digital interviews and testing (Woods et al., 2020). Among these approaches, game-related assessments (GRAs) have recently attracted significant attention (Landers & Sanchez, 2022). Their popularity is mainly based on the idea that they reduce the risk of faking and improve applicants' reactions while maintaining predictive validity (Melchers & Basch, 2022;Wu et al., 2022). However, research on GRAs invites skepticism and seeking more evidence (Ramos-Villagrasa et al., 2022). This study extends prior literature by focusing on a type of, until now, scarcely researched type of GRA (i.e., game-based assessments), considering four job performance dimensions (i.e., task performance, contextual performance, counterproductive work behaviors, and adaptive performance). It also analyzes applicants' reactions to different issues (i.e., perception of comfort and suitability) and investigates the influence of individual differences in scores and reactions to game-based assessments. GRAs: Types and CharacteristicsGRAs refer to the application of gamification science to assessment contexts. Gamification science allows the use of game elements in non-gaming contexts such as personnel selection or training (Landers

show abstract

Intelligence at play: game-based assessment using a virtual-reality application

Cited by 4 publications

References 66 publications

Are serious games an alternative to traditional personality questionnaires? Initial analysis of a gamified assessment

Are serious games an alternative to traditional personality questionnaires? Initial analysis of a gamified assessment

Checking Different Video Game Mechanics to Assess Cognitive Abilities in Groups with and without Emotional Problems

Predictive Validity, Applicant Reactions, and Influence of Personal Characteristics of a Gamefully Designed Assessment

Contact Info

Product

Resources

About