Smart Exam Evaluator for Object-Oriented Programming Modules

Wickramasinghe, M. L.; Wijethunga, H.P; Yapa, Sanjaya; Vishwajith, D. M. D.; Arachchillage, Udara Srimath S. Samaratunge; Amarasena, Nelum

doi:10.1109/icac51239.2020.9357320

Cited by 2 publications

(2 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Very few papers discussed multiple tools, and for these papers, we annotated based on all the tools discussed, which were typically within the same domain as each other. Furthermore, few tools were discussed in multiple papers, with only Antlr [78,132,154,181,190], VPL [90,91,125], Coderunner [101,109,191] and Travis [88,122,123] appearing in three or more papers, with Antlr and Travis primarily being underlying technologies for AATs. While there is some duplication in terms of papers discussing multiple tools and tools being discussed in multiple papers, this duplication does not overly skew the results of the literature review.…”

Section: Resultsmentioning

confidence: 99%

“…Figure 10 shows the count of tools by different evaluation techniques that involved comparing the results from the AAT with a human and the authors' sentiment of the performance of the AAT. Most are evaluated against the accuracy of the AAT compared to human-provided grades [79,108,166,172,181,185,186,190], with most reporting positive results.…”

Section: Performance Against Human Graders (Rq4 Rq5)mentioning

confidence: 99%

See 1 more Smart Citation

Automated Grading and Feedback Tools for Programming Education: A Systematic Review

Messer,

Brown,

Kölling

et al. 2024

ACM Trans. Comput. Educ.

View full text Add to dashboard Cite

We conducted a systematic literature review on automated grading and feedback tools for programming education. We analysed 121 research papers from 2017 to 2021 inclusive and categorised them based on skills assessed, approach, language paradigm, degree of automation and evaluation techniques. Most papers assess the correctness of assignments in object-oriented languages. Typically, these tools use a dynamic technique, primarily unit testing, to provide grades and feedback to the students or static analysis techniques to compare a submission with a reference solution or with a set of correct student submissions. However, these techniques’ feedback is often limited to whether the unit tests have passed or failed, the expected and actual output, or how they differ from the reference solution. Furthermore, few tools assess the maintainability, readability or documentation of the source code, with most using static analysis techniques, such as code quality metrics, in conjunction with grading correctness. Additionally, we found that most tools offered fully automated assessment to allow for near-instantaneous feedback and multiple resubmissions, which can increase student satisfaction and provide them with more opportunities to succeed. In terms of techniques used to evaluate the tools’ performance, most papers primarily use student surveys or compare the automatic assessment tools to grades or feedback provided by human graders. However, because the evaluation dataset is frequently unavailable, it is more difficult to reproduce results and compare tools to a collection of common assignments.

show abstract