Experience with a computer-assisted formal programming examination

Abstract

This paper describes a web-based system for the online delivery of formal examinations and their automated marking. This system was first used in June 2001 in an end-of-year exam for a first year undergraduate programming course. The outcome of this experiment is also described.

How shall we assess this?

“…He regularly uses online practical programming examinations for first-year students [31]. They are conducted under normal closed-book examination conditions; however, many of the questions involve writing program fragments that are marked electronically after the end of the examination.…”

“…The number of papers related to the topic that have been presented at ITiCSE conferences in recent years [e.g. 21,30,31,41,49,65] reflects this increasing interest. Automated assessment can save time and human resources but its adoption must be pedagogically sound.…”

“…CAA can also be used to test the analysis level. For example, by providing students with a program containing bugs and requiring them to correct it into a program that compiles and functions correctly [31]. There are also several integrated systems and tools for assessing complete programming assignments.…”

“…It can be deduced from the survey that it is more common to combine CAA use with manual marking than to rely on it totally for fully electronic marking; for example either flagging all cases that are not clearly correct for human inspection [31] or complementing the automatic assessment with human assessment [1,46]. According to survey respondents, people use CAA more for summative assessment than formative assessment, although it is used for both.…”

The visual authoring tool of flash-based component for interactive item template

“…However, item templates provided by the auto-grading on-line testing systems are plainly true & false, multiple choice (choose one right), multiple selection (choose all right). Some templates, though presented with multimedia statements and options, fail to provide abundant interactive answering alternatives such as click and select, drag and drop, link, collide and magnetize [4] [5]. For scientific knowledge which is in need of incorporating interactive answering methods, current on-line testing system was found lacking better solutions for the more operative-oriented assessment [1].…”