Background. Definitions of gamification tend to vary by person, both in industry and within academia. One particularly popular lay interpretation, introduced and popularized by Ian Bogost, and reiterated by Jan Klabbers, is that gamification is “bullshit” and “exploitationware.” They describe gamification as a marketing term or business practice invented to sell products rather than to represent a real and unique phenomenon relevant to a nascent game science. However, this view is an oversimplification, one which ignores a growing body of theory development and empirical research on gamification within a post-positivist epistemology. In fact, because gamification is so much more outcome-focused than general game design, current gamification research in many ways has a stronger footing in modern social science than much games research does. Aim. In this article, to address common misunderstandings like these, we describe the philosophical underpinnings of modern gamification research, define the relationship between games and gamification, define and situate gamification science as a subdiscipline of game science, and explicate a six-element framework of major concerns within gamification science: predictor constructs, criterion constructs, mediator constructs, moderator constructs, design processes, and research methods. This framework is also presented diagrammatically as a causal path model. Conclusion. Gamification science refers to the development of theories of gamification design and their empirical evaluation within a post-positivist epistemology. The goal of gamification scientist-practitioners should be to understand how to best meet organizational goals through the design of gamification interventions, drawing upon insights derived from both gamification science and games research more broadly.

PurposeAssessment gamification, which refers to the addition of game elements to existing assessments, is commonly implemented to improved applicant reactions to existing psychometric measures. This study aims to understand the effects of gamification on applicant reactions to and measurement quality of situational judgment tests.Design/methodology/approachIn a 2 × 4 between-subjects experiment, this study randomly assigned 315 people to experience different versions of a gamified situational judgment test, crossing immersive game elements (text, audio, still pictures, video) with control game elements (high and low), measuring applicant reactions and assessing differences in convergent validity between conditions.FindingsThe use of immersive game elements improved perceptions of organizational technological sophistication, but no other reactions outcomes (test attitudes, procedural justice, organizational attractiveness). Convergent validity with cognitive ability was not affected by gamification.Originality/valueThis is the first study to experimentally examine applicant reactions and measurement quality to SJTs based upon the implementation of specific game elements. It demonstrates that small-scale efforts to gamify assessments are likely to lead to only small-scale gains. However, it also demonstrates that such modifications can be done without harming the measurement qualities of the test, making gamification a potentially useful marketing tool for assessment specialists. Thus, this study concludes that utility should be considered carefully and explicitly for any attempt to gamify assessment.

In the context of game-based assessments (GBAs), we examined the potential of trace data modeling to supplement or replace an existing GBA's scoring approach.We used data science tools and "big data" practices, such as feature engineering and a series of machine learning algorithms, to predict traditionally measured cognitive ability and conscientiousness scores from the copious trace data generated by a theory-driven GBA designed to measure cognitive ability. Several types of predictors were developed from the raw trace data from 621 participants, including counts of game objects that the player interacted with, the amount of time spent doing so, and mouse movement data across a variety of meaningful intervals. Broadly, we found promising evidence for trace data modeling of cognitive ability, including incremental contribution to the prediction of a criterion grade point average (GPA), but less promising evidence for trace data modeling of conscientiousness, suggesting that trace data modeling like this may be more valuable for assessing traits in games that were developed to target those traits.

In the social and cognitive sciences, crowdsourcing provides up to half of all research participants. Despite this popularity, researchers typically do not conceptualize participants accurately, as gig-economy worker-participants. Applying theories of employee motivation and the psychological contract between employees and employers, we hypothesized that pay and pay raises would drive worker-participant satisfaction, performance, and retention in a longitudinal study. In an experiment hiring 359 Amazon Mechanical Turk Workers, we found that initial pay, relative increase of pay over time, and overall pay did not have substantial influence on subsequent performance. However, pay significantly predicted participants' perceived choice, justice perceptions, and attrition. Given this, we conclude that worker-participants are particularly vulnerable to exploitation, having relatively low power to negotiate pay. Results of this study suggest that researchers wishing to crowdsource research participants using MTurk might not face practical dangers such as decreased performance as a result of lower pay, but they must recognize an ethical obligation to treat Workers fairly.