There are many claims that gamification (i.e., using game elements outside games) impact decreases over time (i.e., the novelty effect). Most studies analyzing this effect focused on extrinsic game elements, while fictional and collaborative competition have been recently recommended. Additionally, to the best of our knowledge, no long-term research has been carried out with STEM learners from introductory programming courses (CS1), a context that demands encouraging practice and mitigating motivation throughout the semester. Therefore, the main goal of this work is to better understand how the impact of a gamification design, featuring fictional and competitive-collaborative elements, changes over a 14-week period of time, when applied to CS1 courses taken by STEM students (N = 756). In an ecological setting, we followed a 2x7 quasi-experimental design, where Brazilian STEM students completed assignments in either a gamified or non-gamified version of the same system, which provided the measures (number of attempts, usage time, and system access) to assess user behavior at seven points in time. Results indicate changes in gamification’s impact that appear to follow a U-shaped pattern. Supporting the novelty effect, the gamification’s effect started to decrease after four weeks, decrease that lasted between two to six weeks. Interestingly, the gamification’s impact shifted to an uptrend between six and 10 weeks after the start of the intervention, partially recovering its contribution naturally. Thus, we found empirical evidence supporting that gamification likely suffers from the novelty effect, but also benefits from the familiarization effect, which contributes to an overall positive impact on students. These findings may provide some guidelines to inform practitioners about how long the initial contributions of gamification last, and how long they take to recover after some reduction in benefits. It can also help researchers to realize when to apply/evaluate interventions that use gamification by taking into consideration the novelty effect and, thereby, better understand the real impact of gamification on students’ behavior in the long run.

Students face several difficulties in introductory programming courses (CS1), often leading to high dropout rates, student demotivation, and lack of interest. The literature has indicated that the adequate use of gamification might improve learning in several domains, including CS1. However, the understanding of which (and how) factors influence gamification’s success, especially for CS1 education, is lacking. Thus, there is a clear need to shed light on pre-determinants of gamification’s impact . To tackle this gap, we investigate how user and contextual factors influence gamification’s effect on CS1 students through a quasi-experimental retrospective study ( N = 399), based on a between-subject design (conditions: gamified or non-gamified) in terms of final grade (academic achievement) and the number of programming assignments completed in an educational system (i.e., how much they practised). Then, we evaluate whether and how user and contextual characteristics (such as age, gender, major, programming experience, working situation, internet access, and computer access/sharing) moderate that effect. Our findings indicate that gamification amplified to some extent the impact of practising. Overall, students practising in the gamified version presented higher academic achievement than those practising the same amount in the non-gamified version. Intriguingly, those in the gamified version that practised much more extensively than the average showed lower academic achievements than those who practised comparable amounts in the non-gamified version. Furthermore, our results reveal gender as the only statistically significant moderator of gamification’s effect: in our data, it was positive for females, but nonsignificant for males. These findings suggest which (and how) personal and contextual factors moderate gamification’s effects, indicate the need to further understand and examine context’s role, and show gamification must be cautiously designed to prevent students from playing instead of learning.

No abstract