Architecting an autograder for parallel code

Abstract

As parallel computing grows and becomes an essential part of computer science, tools must be developed to help grade assignments for large courses, especially with the prevalence of Massive Open Online Courses (MOOCs) increasing in recent years. This paper describes some of the general challenges related to building an autograder for parallel code with general suggestions and sample design decisions covering presented assignments. The paper explores the results and experiences from using these autograders to enable the XSEDE 2013 and 2014 Parallel Computing Course using resources from SDSC-Trestles, TACC-Stampede and PSCBlacklight.

An Open edX Extension for Parallel Programming Assignments with Automatic Configurable Grading

“…Carbunescu et.al. presented in [15] a comprehensive analysis of the different issues that arise when trying to implement an autograder for parallel codes. They warned about the challenges posed by, for example, the fact that many parallel programs may produce non identical outputs even if executed on the same hardware and software environment.…”

An Elastic Platform for Large-scale Assessment of Software Assignments for MOOCs (EPLASAM)

“…Carbunescu et al present their results and experiences from using a framework they designed for auto-grading of parallel code for the 2013and 2014XSEDE Parallel Computing Course (Carbunescu et al, 2014. Their platform used a C program to verify the output results and a Python script to manage file handling, job processing, and final grade calculations.…”