Intrinsic elicitation

Abstract

Applied games are increasingly used to collect human subject data such as people's performance or attitudes. Games aford a motive for data provision that poses a validity threat at the same time: as players enjoy winning the game, they are motivated to provide dishonest data if this holds a strategic in-game advantage. Current work on data collection game design doesn't address this issue. We therefore propose a theoretical model of why people provide certain data in games, the Rational Game User Model. We derive a design approach for human subject data collection games that we call Intrinsic Elicitation: data collection should be integrated into the game's mechanics such that honest responding is the necessary, strategically optimal, and least efortful way to pursue the game's goal. We illustrate the value of our approach with a sample analysis of the data collection game Urbanology.

Validity Threats in Quantitative Data Collection With Games: A Narrative Survey

“…In active player communities around games like LEAGUE OF LEGENDS (Riot Games, 2009), for instance, shared views about optimal high-level strategies like character choice (“the meta”) are in constant flux. Complicating the picture further, player actions are regularly shaped by more concerns than mere winning (Gundry & Deterding, 2018). Overall, this means that especially in interdependent multiplayer games and other games with so-called emergent gameplay (sic), gameplay actions and experiences are hard to control and predict on a low level and showcase emergent but again not fully predictable nor controllable patterns on a higher level of organisation.…”

“…However, different game contexts and genres come with different norms how ruthlessly one is allowed and expected to play (Deterding, 2014), and these norms may differ from the situational norms of the activity of context that one wishes to collect data on. For instance, if a game is designed to elicit people’s preferences about different flavours of ice cream, and there is a strategic in-game advantage to answer “chocolate” even if one actually prefers strawberry flavour, the game’s design will confound the responses (see Gundry & Deterding, 2018) for a detailed discussion and design guidelines to mitigate these effects). The focus on winning the game may also override participants’ desire to be a good study subject and lead them to cheat.…”