The Quiver system

Ellsworth, Christopher C.; Fenwick, James B.; Kurtz, Barry L.

doi:10.1145/971300.971374

Cited by 13 publications

(4 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some of them are Online Judge, a system for testing programs in programming contests [68], WebToTeach, a commercial tool which is designed to support a wide variety of programming exercises [69], Quiver, a server for building, maintaining, and administering programming quizzes [70], Praktomat, a system that allows students to read, review, and assess each others programs in order to improve quality and style [71], Web-CAT, a system that encourage students to write their own test cases [72] in order to experience test-first programming and understand the influence of testing on overall software quality [73], Marmoset, a project submission tool with support for automated testing and for collecting code snapshots [74]. In the new, open-source system Marmoset, the feedback that students receive is based on results obtained by testing, while final grading is done by the instructor (after the project's deadline).…”

Section: Tools Based On Automated Testingmentioning

confidence: 99%

Software verification and graph similarity for automated evaluation of students’ assignments

Vujosevic-Janicic

Nikolić

Tošić

et al. 2013

Information and Software Technology

View full text Add to dashboard Cite

a b s t r a c tContext: The number of students enrolled in universities at standard and on-line programming courses is rapidly increasing. This calls for automated evaluation of students assignments. Objective: We aim to develop methods and tools for objective and reliable automated grading that can also provide substantial and comprehensible feedback. Our approach targets introductory programming courses, which have a number of specific features and goals. The benefits are twofold: reducing the workload for teachers, and providing helpful feedback to students in the process of learning. Method: For sophisticated automated evaluation of students' programs, our grading framework combines results of three approaches (i) testing, (ii) software verification, and (iii) control flow graph similarity measurement. We present our tools for software verification and control flow graph similarity measurement, which are publicly available and open source. The tools are based on an intermediate code representation, so they could be applied to a number of programming languages. Results: Empirical evaluation of the proposed grading framework is performed on a corpus of programs written by university students in programming language C within an introductory programming course. Results of the evaluation show that the synergy of proposed approaches improves the quality and precision of automated grading and that automatically generated grades are highly correlated with instructorassigned grades. Also, the results show that our approach can be trained to adapt to teacher's grading style. Conclusions: In this paper we integrate several techniques for evaluation of student's assignments. The obtained results suggest that the presented tools can find real-world applications in automated grading.

show abstract

Section: Tools Based On Automated Testingmentioning

confidence: 99%

Software verification and graph similarity for automated evaluation of students’ assignments

Vujosevic-Janicic

Nikolić

Tošić

et al. 2013

Information and Software Technology

View full text Add to dashboard Cite

show abstract

“…Apart from the complete program, parts of a student program can also be assessed automatically. Methods suggested by Bettini [8] and Quiver system [9] can effectively assess functions and methods of students' Java programs. In WebToTeach [10] and ELP [11], the program is complete just after students fill up the blanks of a given framework with code fragments.…”

Section: A Dynamic Analysismentioning

confidence: 99%

The Recent Development of Automated Programming Assessment

Liang

Liu

et al. 2009

2009 International Conference on Computational Intelligence and Software Engineering

View full text Add to dashboard Cite

Nowadays computer programs are objectively tested and marked by automated programming assessment systems in computer science education. Dynamic analysis and static analysis are two major approaches in the field of automated programming assessment. Methods adopted by dynamic and static analysis have been emphasized and detailed in this paper. Advantages and disadvantages of them implemented in instruction have been analyzed when taking instructional practice into consideration. Although many automated programming assessment systems have been proved to be of great help to both instructors and students in programming instruction, several problems remain unsolved, such as the security problem and algorithms for automatic generation of test data in dynamic analysis, low accuracy and precision of correctness and functionality assessment in static analysis, thus optimal approaches are still under research. The way of effective use of dynamic analysis and static analysis in instructional practice is also suggested in order that instructors can use the results of this study to choose the most appropriate approach in the context of a particular instructional goal. Finally, standardization of automated programming assessment systems, way to open source code and intelligent tutor system for automated programming assessment are expected to come into exist in the future.

show abstract

“…Ellsworth et al describe Quiver [2], an automated QUIz VERification tool that provides a closed-lab environment where students complete a small programming assignment in a limited amount of time. Because Quiver was designed for an interactive, timed environment, it requires that students use a specific client machine and editor, does not provide limited feedback as in release tests, and does not capture fine-grained snapshots of student code.…”

Section: Related Workmentioning

confidence: 99%

Experiences with marmoset

Spacco

Hovemeyer

Pugh

et al. 2006

Proceedings of the 11th Annual SIGCSE Conference on Innovation and Technology in Computer Science Education

View full text Add to dashboard Cite

We developed Marmoset, an automated submission and testing system, to explore techniques to provide improved feedback to both students and instructors as students work on programming assignments, and to collect data to perform detailed research on the development processes of students.To address the issue of feedback, Marmoset provides students with limited access to the results of the instructor's private test cases using a novel token-based incentive system. This both encourages students to start their work early and to think critically about their work. Because students submit early, instructors can monitor all students' progress on test cases, helping identify challenging or ambiguous test cases early in order to update the project specification or devote additional time in lecture or lab sessions to the difficult test cases.To study and better understand the development process of students, Marmoset can be configured to transparently capture snapshots to a central repository everytime students save their files. These detailed development histories offer a unique, detailed perspective of each student's progress on a programming assignment, from the first line of code written and saved all the way through the final edit before the final submission. This type of data has proven extremely valuable many uses, such as mining new bug patterns and evaluating existing bug-finding tools.In this paper, we describe our initial experiences using Marmoset in several introductory computer science courses, from the perspectives of both instructors and students. We also describe some initial research results from analyzing the student snapshot database.

show abstract

The Quiver system

Cited by 13 publications

References 1 publication

Software verification and graph similarity for automated evaluation of students’ assignments

Software verification and graph similarity for automated evaluation of students’ assignments

The Recent Development of Automated Programming Assessment

Experiences with marmoset

Contact Info

Product

Resources

About