Comparing effective and ineffective behaviors of student programmers

Edwards, Stephen H.; Snyder, Jason; Pérez-Quiñones, Manuel A.; Allevato, Anthony; Kim, Dongkwan; Tretola, Betsy

doi:10.1145/1584322.1584325

Cited by 96 publications

(71 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Multiple studies on students' behavior in online learning environments have shown that there is a significant group of students who submit on the last day, and that this behavior impairs performance [30; 31; 32; 33]. Edwards et al [31] found out in a withinsubject study that this is not merely a correlation where a confounding variable, such as talent, causes both the high performance and the tendency to start working earlier.…”

Section: B Side Effects Of Automated Assessmentmentioning

confidence: 99%

The Effect of Achievement Badges on Students’ Behavior: An Empirical Study in a University-Level Computer Science Course

Hakulinen

Auvinen

Korhonen

2015

Int. J. Emerg. Technol. Learn.

168

View full text Add to dashboard Cite

Abstract-Achievement badges are a form of gamification that are used in an attempt to increase user engagement and motivation in various systems. A badge is typically a graphical icon that appears as a reward for the user after reaching an achievement but that has no practical value. In this study, we describe and evaluate the use of achievement badges in the TRAKLA2 online learning environment where students solve interactive, automatically assessed exercises in a Data Structures and Algorithms course throughout the semester. We conducted an experiment where the students (N=281) were randomly divided into a treatment and a control group, with and without achievement badges. Students in the treatment group were awarded achievement badges, for example, for solving exercises correctly on the first attempt, doing exercises early, or solving all the exercises in a round with full points. Grading was the same for both groups, i.e. collecting badges did not affect the final grade, even though the exercise points themselves did. Students' activity in TRAKLA2 was logged in order to find out whether the achievement badges had an effect on their behavior. We also collected numerical and open-ended feedback in order to find out students' attitudes towards the badges. Our results show that achievement badges can be used to affect students' behavior. Statistically significant differences were observed in the time used per exercise, number of sessions, total time, and normalized total number of badges. Furthermore, the majority of the students reported being motivated by the badges. Based on our findings, achievement badges seem to be a promising method to motivate students and to encourage desired study practices.

show abstract

Section: B Side Effects Of Automated Assessmentmentioning

confidence: 99%

The Effect of Achievement Badges on Students’ Behavior: An Empirical Study in a University-Level Computer Science Course

Hakulinen

Auvinen

Korhonen

2015

Int. J. Emerg. Technol. Learn.

168

View full text Add to dashboard Cite

show abstract

“…While these type of systems offer the flexibility to submit assignments at any time or location, there are downsides for using such systems. The lack of human guidance may lead to undesired study practices such as trial and error problem solving [46,90] or returning assignments at the last minute [48,52,55,117].…”

Section: Detailed Research Questionsmentioning

confidence: 99%

QR code programming tasks with automated assessment

Hakulinen

Malmi

2014

Proceedings of the 2014 Conference on Innovation &Amp; Technology in Computer Science Education - ITiCSE '14

View full text Add to dashboard Cite

The main objective of this thesis is to enrich computer science education by studying two gameful teaching interventions: 1) achievement badges and 2) alternate reality games.Gamification is the use of game design elements in non-game contexts, and achievement badges are one commonly used gamification method that can be used to provide optional challenges for students. We studied the use of achievement badges in a Data Structures and Algorithms course and found out that badges can be used to steer students' behavior even if they are not tied to tangible rewards such as course grades. Specifically, students used more time in the online learning environment, had more sessions in the system, and spent more time between exercise submissions when the badges were used. Furthermore, we found out that students' attitudes towards the badges varied. We also studied the use of badges in relation to achievement goal orientations that characterize students' preferences to different goals and outcomes. We found out that students who had high avoidance orientation were less motivated by the badges than other students. On the other hand, students who were the most motivated by the badges had higher mastery-intrinsic, mastery-extrinsic, and performance-approach orientations, and lower avoidance orientation. Furthermore, we compared the badge intervention with the use of heatmaps that provided feedback on one's behavior in a nongameful way. An interest towards the badges correlated with mastery-extrinsic and performance-approach, whereas interest towards the heatmaps correlated with masteryextrinsic and performance-avoidance orientations.Alternate Reality Games (ARG), are games that blur the line between reality and fiction and typically they involve puzzles and an interactive narrative. We explored the use of ARGs for teaching computer science by organizing an ARG covering a wide range of computer science concepts. In the feedback collected after the game, participants reported learning several computer science concepts during the game. We also evaluated how authenticity, intrinsic motivation, and replayability were taken into account in the game design. Furthermore, we present an example of an educational ARG that was not part of any official curriculum and relied on voluntary participation. Based on the results, alternate reality games seem like a promising approach to be used in computer science education.In addition to evaluating the two approaches, we provide support for applying them in education by describing the implementation of our badge systems and introducing a method for producing automatically assessed programming tasks that are suitable for ARGs. Tiivistelmä Väitöskirjassa tarkastellaan kahta pelillistä tapaa monimuotoistaa tietotekniikan opetusta: 1) suoritusmerkit ja 2) vaihtoehtotodellisuuspelit.Pelillistämisellä tarkoitetaan pelielementtien soveltamista eri ympäristöissä. Suoritusmerkit ovat eräs yleinen pelillistämisen keino, jolla voidaan tarjota opiskelijoille vapaaehtoisia lisä-haasteita. Väitöskir...

show abstract

“…[25] give an overview of early programming behavior studies. Programming behavior has been analyzed using various types of data, including video analysis [19], [35], analysis of graded programming assignments [11] or students' newsgroup discussions [18]. [29] present a classification of source code search strategies.…”

Section: Related Workmentioning

confidence: 99%

“…"How to call a Java class from a Android Activity" 9 , "Android: AsyncTask recommen- dations: private class or public class?" 10 , "Android: 'thread exiting with uncaught exception (group=0x4001d88)'" 11 . This helps us explain why the students accessed 87 different questions in the 105 results, of which 75 were accessed only once.…”

Section: Browser Log Analysismentioning

confidence: 99%

Monitoring students' mobile app coding behavior data analysis based on IDE and browser interaction logs

Fuchs

Heckner

Raab

et al. 2014

2014 IEEE Global Engineering Education Conference (EDUCON)

View full text Add to dashboard Cite

This paper describes a case study of assessing student's coding behavior and skills in a realistic development setting. Students had to solve typical programming problems in the context of app development for the Android platform using the Eclipse IDE. Data was analyzed using IDE as well as browser interaction logs. In addition, screen recordings of the students' interaction with the IDE provide further insight. In this paper we present the first results of our ongoing work.

show abstract

Comparing effective and ineffective behaviors of student programmers

Cited by 96 publications

References 7 publications

The Effect of Achievement Badges on Students’ Behavior: An Empirical Study in a University-Level Computer Science Course

The Effect of Achievement Badges on Students’ Behavior: An Empirical Study in a University-Level Computer Science Course

QR code programming tasks with automated assessment

Monitoring students' mobile app coding behavior data analysis based on IDE and browser interaction logs

Contact Info

Product

Resources

About