Towards automated assessment in game-creative programming courses

Pietriková, Emília; Juhár, Ján; Stastna, Jana

doi:10.1109/iceta.2015.7558505

Cited by 10 publications

(7 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Tool / Reference Type Language Availability Interoperability URL Web-CAT [75] WP Multi OS LTI + Eclipse + BlueJ https://web-cat.github.io/ PASS [255] WP C/C++, Java A https://www.cs.cityu.edu.hk/~passweb Kattis [77] WP Multi A https://open.kattis.com ECSpooler [13] WS Multi OS https://github.com/collective/ECSpooler STAGE (CodeCover) [172] PG Java, COBOL OS Moodle plugin http://codecover.org JACK [30,136,227] WP Multi A LTI + Eclipse https://jack-demo.s3.uni-due.de URI Online Judge [32] WP Multi A https://www.urionlinejudge.com.br CodeAssessor [241] WP C/C++ VPL [203] PG Multi OS Moodle plugin https://vpl.dis.ulpgc.es/ CloudCoder [108] WP C/C++, Java, Python, Ruby OS https://github.com/daveho/CloudCoder CodeRunner [60] PG Multi OS Moodle plugin https://github.com/trampgeek/moodle-qtype_coderunner JPLAS [83][84][85] WP 1 Java Eclipse LX [61] WS Multi Modiied Moodle Testovid within Protus [242] WP Java OS https://github.com/ivanpribela/testovid TestMyCode [194] WP Multi OS NetBeans https://github.com/testmycode/tmc-server CodeWorkout [74] WP C++, Java, Python, Ruby OS LTI https://github.com/web-cat/code-workout Arena [184] WP Imp/OO Git JavAssess (ASys) [115] LB Java OS http://personales.upv.es/josilga/ASys CodeOcean [226] WP Multi OS LTI https://github.com/openHPI/codeocean Jutge.org [176] WP Multi OS https://github.com/jutge-org GradeIT (PrUTOR) [62,173] CD C A https://prutor.ai neoESPA [148,150] WP C/C++, Java, Python A http://neoespa.pusan.ac.kr Enki (Mooshak) [170] WP Multi OS LTI + Eclipse https://mooshak2.dcc.fc.up.pt ProgEdu [50,252] CD Java OS Git https://github.com/ProgEdu-Organization/ProgEdu Aristotle [2] ...…”

Section: Table 3 General Information About the Selected Toolsmentioning

confidence: 99%

“…Simple statistics comprises organizing, displaying, and summarizing data from server interactions (e.g., login activity, submissions, obtained grades and classiications, etc.). In contrast, advanced analysis involves drawing conclusions about the data as well ✓ ✓ OC ECSpooler [13] ✓ OC + UT STAGE (CodeCover) [172] ✓ ✓(Coverage) UT JACK [30,136,227] ✓ ✓ ✓ OC ✓ URI Online Judge [32] ✓ ✓ OC ✓ CodeAssessor [241] ✓ OC VPL [203] ✓ ✓ OC ✓ CloudCoder [108] ✓ OC CodeRunner [60] ✓ OC JPLAS [83][84][85] ✓ UT LX [61] ✓ OC + UT Testovid within Protus [242] ✓ ✓(Checkstyle) ✓ UT TestMyCode [194] ✓ ✓(Checkstyle) UT ✓ CodeWorkout [74] ✓ UT Arena [184] ✓ UT JavAssess (ASys) [115] ✓ JR OC JR ✓ CodeOcean [226] ✓ OC + UT ✓ Jutge.org [176] ✓ ✓ OC 2 ✓ GradeIT (PrUTOR) [62,173]…”

Section: Table 3 General Information About the Selected Toolsmentioning

confidence: 99%

See 1 more Smart Citation

Automated Assessment in Computer Science Education: A State-of-the-Art Review

Paiva

Leal

Figueira

2022

ACM Trans. Comput. Educ.

108

View full text Add to dashboard Cite

Practical programming competencies are critical to the success in computer science education and go-to-market of fresh graduates. Acquiring the required level of skills is a long journey of discovery, trial and error, and optimization seeking through a broad range of programming activities that learners must perform themselves. It is not reasonable to consider that teachers could evaluate all attempts that the average learner should develop multiplied by the number of students enrolled in a course, much less in a timely, deeply, and fairly fashion. Unsurprisingly, exploring the formal structure of programs to automate the assessment of certain features has long been a hot topic among CS education practitioners. Assessing a program is considerably more complex than asserting its functional correctness, as the proliferation of tools and techniques in the literature over the past decades indicates. Program efficiency, behavior, readability, among many other features, assessed either statically or dynamically, are now also relevant for automatic evaluation. The outcome of an evaluation evolved from the primordial boolean values to information about errors and tips on how to advance, possibly taking into account similar solutions. This work surveys the state-of-the-art in the automated assessment of CS assignments, focusing on the supported types of exercises, security measures adopted, testing techniques used, type of feedback produced, and the information they offer the teacher to understand and optimize learning. A new era of automated assessment, capitalizing on static analysis techniques and containerization, has been identified. Furthermore, this review presents several other findings from the conducted review, discusses the current challenges of the field, and proposes some future research directions.

show abstract

Section: Table 3 General Information About the Selected Toolsmentioning

confidence: 99%

Section: Table 3 General Information About the Selected Toolsmentioning

confidence: 99%

Automated Assessment in Computer Science Education: A State-of-the-Art Review

Paiva

Leal

Figueira

2022

ACM Trans. Comput. Educ.

108

View full text Add to dashboard Cite

show abstract

“…Domain-specific languages are successfully used in different areas, for example, application logging [13], acceptance tests [14], graphic shape description [15], expert systems [16], text analysis [17], or automatic assessment of students [18]. Source code annotations can also be considered a DSLs [19].…”

Section: Related Workmentioning

confidence: 99%

Abstract-Syntax-Driven Development of Oberon-0 Using YAJCo

Chodarev

Bacikova

2019

J. inf. organ. sci. (Online)

View full text Add to dashboard Cite

YAJCo is a tool for the development of software languages based on an annotated language model. The model is represented by Java classes with annotations defining their mapping to concrete syntax. This approach to language definition enables the abstract syntax to be the central point of the development process, instead of concrete syntax. In this paper, a case study of Oberon-0 programming language development is presented. The study is based on the LTDA Tool Challenge and showcases details of abstract and concrete syntax definition using YAJCo, as well as the implementation of name resolution, type checking, model transformation, and code generation. The language was implemented in a modular fashion to demonstrate language extension mechanisms supported by YAJCo.

show abstract

“…It usually consists of standard output, error output and return code of a testcase command. More details on problemset package preparation and presentation of submission outputs are published in (Pietriková, Juhár & Šťastná, 2015).…”

Section: Services Of the Arena Platformmentioning

confidence: 99%

“…With high numbers of attendees it is crucial to provide fast feedback and fair assessment of assignments, which is not possible to achieve without automation of the assessment process. The educational view on the evaluation of programming assignments is also discussed in a work of Biňas and Pietriková (2014) and Pietriková, Juhár, and Šťastná (2015).…”

Section: Introductionmentioning

confidence: 99%

Security Measures in Automated Assessment System for Programming Courses

Stastna¹,

Juhár²,

Binas³

et al. 2015

AIP

View full text Add to dashboard Cite

A desirable characteristic of programming code assessment is to provide the learner the most appropriate information regarding the code functionality as well as a chance to improve. This can be hardly achieved in case the number of learners is high (500 or more). In this paper we address the problem of risky code testing and availability of an assessment platform Arena, dealing with potential security risks when providing an automated assessment for a large set of source code. Looking at students' programs as if they were potentially malicious inspired us to investigate separated execution environments, used by security experts for secure software analysis. The results also show that availability issues of our assessment platform can be conveniently resolved with task queues. A special attention is paid to Docker, a virtual container ensuring no risky code can affect the assessment system security. The assessment platform Arena enables to regularly, effectively and securely assess students' source code in various programming courses. In addition to that it is a motivating factor and helps students to engage in the educational process.

show abstract

Towards automated assessment in game-creative programming courses

Cited by 10 publications

References 11 publications

Automated Assessment in Computer Science Education: A State-of-the-Art Review

Automated Assessment in Computer Science Education: A State-of-the-Art Review

Abstract-Syntax-Driven Development of Oberon-0 Using YAJCo

Security Measures in Automated Assessment System for Programming Courses

Contact Info

Product

Resources

About