Analyzing Student Work Patterns Using Programming Exercise Data

Spacco, Jaime; Denny, Paul; Richards, Brad; Babcock, David; Hovemeyer, David; Moscola, James; Duvall, Robert C.

doi:10.1145/2676723.2677297

Cited by 62 publications

(29 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Overall, in line with previous studies, we observed that students' performance decreases during the course. This is in line with both the general knowledge in introductory programming as well as the literature where students' performance in online programming environment has been analyzed [18]. However, when contrasting our results to those in [18], students in our context are working in a traditional programming environment, and the majority of the course assignments expect that the students implement more than a single function -thus, generalizing the previous results to a new context.…”

Section: Discussion 51 Overall Performance In the Coursesupporting

confidence: 89%

“…With research question one, we aim to both validate previous results by Spacco et al [18] in a new context, and to extend the work by analyzing students' performance changes from week to week. These changes are discussed in the light of the content taught in the course.…”

Section: Research Design 31 Research Questionsmentioning

confidence: 85%

“…In-class behavior has been explored by Porter et al [15] who observed that the proportion of correct clicker answers early in a course was strongly correlated with final course outcomes. Similarly, interaction with web-based programming environments has been studied [18]; Spacco et al noticed statistically significant but weak relationships between the final exam score and students' effort.…”

Section: Work Patterns and Effortmentioning

confidence: 99%

See 2 more Smart Citations

Performance and Consistency in Learning to Program

Ahadi

Lister

Lal

et al. 2017

Proceedings of the Nineteenth Australasian Computing Education Conference

View full text Add to dashboard Cite

Performance and consistency play a large role in learning. Decreasing the effort that one invests into course work may have short-term benefits such as reduced stress. However, as courses progress, neglected work accumulates and may cause challenges with learning the course content at hand. In this work, we analyze students' performance and consistency with programming assignments in an introductory programming course. We study how performance, when measured through progress in course assignments, evolves throughout the course, study weekly fluctuations in students' work consistency, and contrast this with students' performance in the course final exam. Our results indicate that whilst fluctuations in students' weekly performance do not distinguish poor performing students from well performing students with a high accuracy, more accurate results can be achieved when focusing on the performance of students on individual assignments which could be used for identifying struggling students who are at risk of dropping out of their studies.

show abstract

Section: Discussion 51 Overall Performance In the Coursesupporting

confidence: 89%

Section: Research Design 31 Research Questionsmentioning

confidence: 85%

Section: Work Patterns and Effortmentioning

confidence: 99%

See 1 more Smart Citation

Performance and Consistency in Learning to Program

Ahadi

Lister

Lal

et al. 2017

Proceedings of the Nineteenth Australasian Computing Education Conference

View full text Add to dashboard Cite

show abstract

“…Often, these studies are based upon snapshots of the students' code and recordings of their activities, investigating how these habits correlate with the students' performance. Several researchers have noted that lower performing students usually take longer to complete the exercises [1], [55], [52]. Prior research that has tracked student activities during their coding sessions has also shown that better performing students usually analyze the code in a more logical manner and spot the issues faster, while lower performing students have difficulty localizing problems [55], [43].…”

Section: Social Learning and Study Habitsmentioning

confidence: 99%

How Widely Can Prediction Models Be Generalized? Performance Prediction in Blended Courses

Gitinabard

Heckman

et al. 2019

IEEE Trans. Learning Technol.

View full text Add to dashboard Cite

Blended courses that mix in-person instruction with online platforms are increasingly popular in secondary education. These tools record a rich amount of data on students' study habits and social interactions. Prior research has shown that these metrics are correlated with students' performance in face to face classes. However, predictive models for blended courses are still limited and have not yet succeeded at early prediction or cross-class predictions even for repeated offerings of the same course. In this work, we use data from two offerings of two different undergraduate courses to train and evaluate predictive models on student performance based upon persistent student characteristics including study habits and social interactions. We analyze the performance of these models on the same offering, on different offerings of the same course, and across courses to see how well they generalize. We also evaluate the models on different segments of the courses to determine how early reliable predictions can be made. This work tells us in part how much data is required to make robust predictions and how cross-class data may be used, or not, to boost model performance. The results of this study will help us better understand how similar the study habits, social activities, and the teamwork styles are across semesters for students in each performance category. These trained models also provide an avenue to improve our existing support platforms to better support struggling students early in the semester with the goal of providing timely intervention.

show abstract

“…Spacco [2] collected student programming data across five semesters and three institutions. Spacco found that as students do incrementally-harder exercises, students' scores tended to decrease, whereas the chance of submitting correctly compiling code tended to increase.…”

Section: Researchers [1][2][5][9]mentioning

confidence: 99%

An Analysis of Incorporating Small Coding Exercises as Homework in Introductory Programming Courses

Edgcomb

Vahid

Lysecky³

et al.

2017 ASEE Annual Conference &Amp; Exposition Proceedings

View full text Add to dashboard Cite

Alex Edgcomb finished his PhD in computer science at UC Riverside in 2014. Alex works with zyBooks.com, a startup that develops interactive, web-native textbooks in STEM. Alex also works as a research specialist at UC Riverside, studying the efficacy of web-native content for STEM education. His research interests include embedded systems, runtime optimization, non-intrusive system observation methods, data-adaptable systems, and embedded system security. He has recently coauthored multiple textbooks, published by zyBooks, that utilize a web-native, interactive, and animated approach, which has shown notable increases in student learning and course grades. Prof ABSTRACTSmall auto-graded coding exercises with immediate feedback are widely recognized as helping students in introductory programming courses. We analyzed usage of a particular coding exercise tool, at 11 courses at different universities. Instructors awarded differing amounts of course points (including zero) for the exercises. We investigated how awarding points affected student completion rates of the exercises. We found that without awarding points, completion rates were about 25%. Awarding even a small amount of points, such as 2 course points out of 100, resulted in 62% completion, with little increase in completion rates for more course points (such as 5, 10, or even 25). Comparing to participation activity completion rates of 85%, one might conclude that the 62% is short of 100% in part due to some students simply not doing homework (15%), and the remaining 23% due to the greater difficulty of the exercises. We analyzed time spent, and found that students spent about 3.3 minutes per exercise, matching the expected 2-4 minutes by the exercise authors. We analyzed number of tries per exercise, and found students submitted 3.5 tries on average per exercise. For some harder exercises, the averages were higher at 5-10 tries, suggesting the students are indeed putting forth good effort. We found very high numbers of tries by some students on a single exercise, sometimes 30, 50, or even 100, suggesting more work is needed to assist such students to better learn the concepts rather than repeatedly submitting tries, and to reduce frustration and increase learning efficiency.

show abstract

Analyzing Student Work Patterns Using Programming Exercise Data

Cited by 62 publications

References 14 publications

Performance and Consistency in Learning to Program

Performance and Consistency in Learning to Program

How Widely Can Prediction Models Be Generalized? Performance Prediction in Blended Courses

An Analysis of Incorporating Small Coding Exercises as Homework in Introductory Programming Courses

Contact Info

Product

Resources

About