Raymond S. Pettit teaches courses in programming, artificial intelligence, objected oriented design, algorithms, theory of computation, and related subjects in ACU's School of Information Technology and Computing. Prior to joining the ACU faculty, he spent twenty years in software development, research, and training the Air Force Research Lab and NASA's Langley Research Center as well as private industry. His current research focuses on how automated assessment tools interact with student learning in university programming courses.
Are Automated Assessment Tools Helpful in ProgrammingCourses?
AbstractAutomated assessment tools (AATs) are growing in popularity in introductory programming courses, but researchers may have a difficult time synthesizing valid data to draw conclusions about the tools' usefulness. Our first step addressing this issue was to break down our overriding question-are automated assessment tools helpful in programming courses?-into four more specific questions: (1) Have AATs proven to be helpful in improving student learning? (2) Do students think that AATs have improved their performance? (3) After having used the tools, do instructors think that the tools have improved their teaching experiences? and (4) Is the assessment performed by AATs accurate enough to be helpful? In discussing the many AATs that exist, many researchers have only reported results relevant to one or two of these specific questions. We address each of our four questions separately and draw on data from 24 different tools to arrive at our conclusions. We determine that the literature demonstrates AATs helpfulness in student learning, instructor support, and assessment accuracy. However, we found results about students' opinions regarding the helpfulness of AATs to be inconclusive. Given our findings, we make suggestions both for instructors using these tools and to researchers creating them.