Automata Tutor v3

Abstract

Computer science class enrollments have rapidly risen in the past decade. With current class sizes, standard approaches to grading and providing personalized feedback are no longer possible and new techniques become both feasible and necessary. In this paper, we present the third version of Automata Tutor, a tool for helping teachers and students in large courses on automata and formal languages. The second version of Automata Tutor supported automatic grading and feedback for finiteautomata constructions and has already been used by thousands of users in dozens of countries. This new version of Automata Tutor supports automated grading and feedback generation for a greatly extended variety of new problems, including problems that ask students to create regular expressions, context-free grammars, pushdown automata and Turing machines corresponding to a given description, and problems about converting between equivalent models-e.g., from regular expressions to nondeterministic finite automata. Moreover, for several problems, this new version also enables teachers and students to automatically generate new problem instances. We also present the results of a survey run on a class of 950 students, which shows very positive results about the usability and usefulness of the tool.

Theoretical Computer Science in Basic Education: A Systematic Review

“…A moodle-supported online course on finite automata, push down automata and turing machines [Chuda 2007]. An automatic problem-generator tool to create exercises for students learning automatons [D'Antoni et al 2020]. A game design activity using formal languages and regular expressions to build finite state and turing machines modeling games [Korte et al 2007].…”

Automated Grading of Regular Expressions

“…We recruited 20 undergraduate students who were taking or had taken an automata course at the time of conducting our research, and ran our automatic grading algorithm on students' regex submissions for ten selected exercises from famous automata textbooks [9,14,18]. In order to compare the results of automated grading with the previous approaches including RegED [10] and AT v3 [5], we implemented the algorithms in Python 3 on our own and used them for comparison. We cannot use the existing implementations directly, because they do not support a feature of adjusting the maximum number of allowed edits, and not all of them are supported as a tool.…”

“…However, both works suffer from a limitation that 'optimal' answers for the problems should be given by TAs as they compare the students' submissions with the answers for giving partial grades. Recently, D'Antoni et al [5] propose Automata Tutor v3 (abbreviated to AT v3 hereafter), which is the latest version of the previous work [2]. In AT v3, they include automated grading and feedback generation for a variety of new automata problems including the problems that ask to create regexes, context-free grammars, pushdown automata, and even Turing machines for a given description (e.g., a natural language description, or an automaton, or a grammar that belongs to a different class).…”

“…Performance comparisons of the proposed grading algorithm with baseline algorithms proposed in previous works[5,10].…”

Automated Grading of Automata with ACL2s

“…Instructors can use ACL2s functionality to specify and check properties over these models, or use the full power of the ACL2s theorem prover to implement custom checks. This advantage is usually missing from visual or XML based representations, such as those used in Automata Tutor V3 [13] or JFLAP [25]. Furthermore, access to a powerful theorem prover allows an instructor to test their constructions, prove properties or even add new theories or models.…”