An Extension of Code Correction Problem for Java Programming Learning Assistant System

Funabiki, Nobuo; Kuribayashi, Minoru; Kyaw, Htoo; Wint, Su Sandy; Aung, Soe Thandar; Kao, Wen-Chung

doi:10.1145/3395245.3396439

Cited by 2 publications

(5 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Existing tools supporting automated assessment tend to overlook the instructor's perspective, only providing a few mechanisms to collect and visualize learners' data, such as submissions' history, evaluation reports, and completed activities [40,85]. Other tools [151,164,203,226] integrate with an LMS to ofer the instructor a familiar dashboard which, even though it has several analytical and reporting tools for learning [253], has no means to work with the speciic data produced in computer programming.…”

Section: Learning Analytics (Rq5)mentioning

confidence: 99%

“…Tool / Reference Type Language Availability Interoperability URL Web-CAT [75] WP Multi OS LTI + Eclipse + BlueJ https://web-cat.github.io/ PASS [255] WP C/C++, Java A https://www.cs.cityu.edu.hk/~passweb Kattis [77] WP Multi A https://open.kattis.com ECSpooler [13] WS Multi OS https://github.com/collective/ECSpooler STAGE (CodeCover) [172] PG Java, COBOL OS Moodle plugin http://codecover.org JACK [30,136,227] WP Multi A LTI + Eclipse https://jack-demo.s3.uni-due.de URI Online Judge [32] WP Multi A https://www.urionlinejudge.com.br CodeAssessor [241] WP C/C++ VPL [203] PG Multi OS Moodle plugin https://vpl.dis.ulpgc.es/ CloudCoder [108] WP C/C++, Java, Python, Ruby OS https://github.com/daveho/CloudCoder CodeRunner [60] PG Multi OS Moodle plugin https://github.com/trampgeek/moodle-qtype_coderunner JPLAS [83][84][85] WP 1 Java Eclipse LX [61] WS Multi Modiied Moodle Testovid within Protus [242] WP Java OS https://github.com/ivanpribela/testovid TestMyCode [194] WP Multi OS NetBeans https://github.com/testmycode/tmc-server CodeWorkout [74] WP C++, Java, Python, Ruby OS LTI https://github.com/web-cat/code-workout Arena [184] WP Imp/OO Git JavAssess (ASys) [115] LB Java OS http://personales.upv.es/josilga/ASys CodeOcean [226] WP Multi OS LTI https://github.com/openHPI/codeocean Jutge.org [176] WP Multi OS https://github.com/jutge-org GradeIT (PrUTOR) [62,173] CD C A https://prutor.ai neoESPA [148,150] WP C/C++, Java, Python A http://neoespa.pusan.ac.kr Enki (Mooshak) [170] WP Multi OS LTI + Eclipse https://mooshak2.dcc.fc.up.pt ProgEdu [50,252] CD Java OS Git https://github.com/ProgEdu-Organization/ProgEdu Aristotle [2] ...…”

Section: Table 3 General Information About the Selected Toolsmentioning

confidence: 99%

“…Simple statistics comprises organizing, displaying, and summarizing data from server interactions (e.g., login activity, submissions, obtained grades and classiications, etc.). In contrast, advanced analysis involves drawing conclusions about the data as well ✓ ✓ OC ECSpooler [13] ✓ OC + UT STAGE (CodeCover) [172] ✓ ✓(Coverage) UT JACK [30,136,227] ✓ ✓ ✓ OC ✓ URI Online Judge [32] ✓ ✓ OC ✓ CodeAssessor [241] ✓ OC VPL [203] ✓ ✓ OC ✓ CloudCoder [108] ✓ OC CodeRunner [60] ✓ OC JPLAS [83][84][85] ✓ UT LX [61] ✓ OC + UT Testovid within Protus [242] ✓ ✓(Checkstyle) ✓ UT TestMyCode [194] ✓ ✓(Checkstyle) UT ✓ CodeWorkout [74] ✓ UT Arena [184] ✓ UT JavAssess (ASys) [115] ✓ JR OC JR ✓ CodeOcean [226] ✓ OC + UT ✓ Jutge.org [176] ✓ ✓ OC 2 ✓ GradeIT (PrUTOR) [62,173]…”

Section: Table 3 General Information About the Selected Toolsmentioning

confidence: 99%

“…1 Token Pattern Approach 2 Several variants that are more tolerant JR -Java relection; UT -unit testing; OC -output comparison Table 5. Security models for untrusted code execution used in the selected tools Tool / Reference Model External Web-CAT [75] JVM (Java) YES PASS [255] User permissions YES Kattis [77] User permissions YES ECSpooler [13] User permissions YES STAGE (CodeCover) [172] JVM YES JACK [30,136,227] JVM YES URI Online Judge [32] User permissions YES CodeAssessor [241] User permissions NO VPL [203] Jail process YES CloudCoder [108] JVM (Java) + Jail process YES CodeRunner [60] User permissions YES JPLAS [83][84][85] Blacklist + JVM NO LX [61] User permissions NO Testovid within Protus [242] JVM YES TestMyCode [194] VM YES CodeWorkout [74] JVM (Java) + User permissions NO Arena [184] Container YES JavAssess (ASys) [115] JVM NO CodeOcean [226] Container YES Jutge.org [176] Jail process YES GradeIT (PrUTOR) [62,173] Container YES neoESPA [148,150] User permissions NO Enki (Mooshak) [170] User permissions NO ProgEdu [50,252] Container YES Aristotle [2] User permissions NO Submitty [180] Container YES Codelex [40] Jail process YES Dante [70] User permissions YES Coderiu [28] Container YES Edgar [156] Container YES Table…”

Section: Table 3 General Information About the Selected Toolsmentioning

confidence: 99%

“…Tool / Reference Failure Info Evaluation Logs Failed Tests How to Proceed Report Manual Web-CAT [75] ✓ ✓ ✓ PASS [255] ✓ ✓ Kattis [77] ✓ ✓ ECSpooler [13] ✓ ✓ STAGE (CodeCover) [172] ✓ ✓ ✓(Coverage) JACK [30,136,227] ✓ ✓ ✓ ✓ URI Online Judge [32] ✓ ✓ CodeAssessor [241] ✓ ✓ VPL [203] ✓ ✓ CloudCoder [108] ✓ ✓ CodeRunner [60] ✓ ✓ JPLAS [83][84][85] ✓ ✓ LX [61] ✓ ✓ Testovid within Protus [242] ✓ ✓ ✓ TestMyCode [194] ✓ ✓ ✓ CodeWorkout [74] ✓ Arena [184] ✓ ✓ ✓ JavAssess (ASys) [115] ✓ ✓ ✓ ✓ ✓ CodeOcean [226] ✓ ✓ ✓ Jutge.org [176] ✓ ✓ ✓ GradeIT (PrUTOR) [62,173] ✓ ✓ ✓ 1 neoESPA [148,150] ✓ ✓(Quality) Enki (Mooshak) [170] ✓ ✓ ✓ ProgEdu [50,252] ✓ ✓ ✓ ✓(Quality) Aristotle [2] Submitty [180] ✓…”

Section: Table 3 General Information About the Selected Toolsmentioning

confidence: 99%

See 4 more Smart Citations

Automated Assessment in Computer Science Education: A State-of-the-Art Review

Paiva

Leal

Figueira

2022

ACM Trans. Comput. Educ.

106

View full text Add to dashboard Cite

Practical programming competencies are critical to the success in computer science education and go-to-market of fresh graduates. Acquiring the required level of skills is a long journey of discovery, trial and error, and optimization seeking through a broad range of programming activities that learners must perform themselves. It is not reasonable to consider that teachers could evaluate all attempts that the average learner should develop multiplied by the number of students enrolled in a course, much less in a timely, deeply, and fairly fashion. Unsurprisingly, exploring the formal structure of programs to automate the assessment of certain features has long been a hot topic among CS education practitioners. Assessing a program is considerably more complex than asserting its functional correctness, as the proliferation of tools and techniques in the literature over the past decades indicates. Program efficiency, behavior, readability, among many other features, assessed either statically or dynamically, are now also relevant for automatic evaluation. The outcome of an evaluation evolved from the primordial boolean values to information about errors and tips on how to advance, possibly taking into account similar solutions. This work surveys the state-of-the-art in the automated assessment of CS assignments, focusing on the supported types of exercises, security measures adopted, testing techniques used, type of feedback produced, and the information they offer the teacher to understand and optimize learning. A new era of automated assessment, capitalizing on static analysis techniques and containerization, has been identified. Furthermore, this review presents several other findings from the conducted review, discusses the current challenges of the field, and proposes some future research directions.

show abstract