The Impact of Iterative Assessment System on Programming Learning Behavior

Yan, Yi-Xiang; Wu, Jungpin; Nguyen, Bao-An; Chen, Hsi-Min

doi:10.1145/3383923.3383939

Cited by 8 publications

(10 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Acceptance of students on code quality improvement functionalities of ProgEdu was surveyed and presented in the article [11]. More than 70% of asked students agreed with the assessment method given by the system, especially with code quality assessment.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Analysis of Learning Behavior in an Automated Programming Assessment Environment: A Code Quality Perspective

et al. 2020

Self Cite

View full text Add to dashboard Cite

Automated programming assessment systems are useful tools to track the learning progress of students automatically and thereby reduce the workload of educators. They can also be used to gain insights into how students learn, making it easier to formulate strategies aimed at enhancing learning performance. Rather than functional code which is always inspected, code quality remains an essential aspect to which not many educators consider when designing an automated programming assessment system. In this study, we applied data mining techniques to analyze the results of an automated assessment system to reveal unexpressed patterns in code quality improvement that are predictive of final achievements in the course. Cluster analysis is first utilized to categorize students according to their learning behavior and outcomes. Cluster profile analysis is then leveraged to highlight actionable factors that could affect their final grades. Finally, the same factors are employed to construct a classification model by which to make early predictions of the students' final results. Our empirical results demonstrate the efficacy of the proposed scheme in providing valuable insights into the learning behaviors of students in novice programming courses, especially in code quality assurance, which could be used to enhance programming performance at the university level. INDEX TERMS automated programming assessment system, code quality, educational data mining, early learning achievement detection, programming education

show abstract

Section: Discussionmentioning

confidence: 99%

“…The effectiveness of ProgEdu has been analyzed through research on 69 students which was presented in our other report [11]. Those analysis results show high acceptance of students on the usage of ProgEdu and the iterative learning approach.…”

Section: Introductionmentioning

confidence: 97%

Analysis of Learning Behavior in an Automated Programming Assessment Environment: A Code Quality Perspective

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…This granularity level in the collection reduces the amount of lost data, whereas its analysis reveals insights that submissions or even snapshots would not [243]. ProgEdu [50,252] keeps a history of changes to the source code using Git and stores logging data to provide instructors with various statistics charts from which they can comprehend whether submissions are on time and meet requirements, the distribution of failure types, and the quality of students' code. PRogramming tUTOR (PRUTOR) [62] takes code snapshots at regular intervals and provides visualizations of the code progression of students' programs and how the students approach the programming assignments.…”

Section: Learning Analytics (Rq5)mentioning

confidence: 99%

“…Tool / Reference Type Language Availability Interoperability URL Web-CAT [75] WP Multi OS LTI + Eclipse + BlueJ https://web-cat.github.io/ PASS [255] WP C/C++, Java A https://www.cs.cityu.edu.hk/~passweb Kattis [77] WP Multi A https://open.kattis.com ECSpooler [13] WS Multi OS https://github.com/collective/ECSpooler STAGE (CodeCover) [172] PG Java, COBOL OS Moodle plugin http://codecover.org JACK [30,136,227] WP Multi A LTI + Eclipse https://jack-demo.s3.uni-due.de URI Online Judge [32] WP Multi A https://www.urionlinejudge.com.br CodeAssessor [241] WP C/C++ VPL [203] PG Multi OS Moodle plugin https://vpl.dis.ulpgc.es/ CloudCoder [108] WP C/C++, Java, Python, Ruby OS https://github.com/daveho/CloudCoder CodeRunner [60] PG Multi OS Moodle plugin https://github.com/trampgeek/moodle-qtype_coderunner JPLAS [83][84][85] WP 1 Java Eclipse LX [61] WS Multi Modiied Moodle Testovid within Protus [242] WP Java OS https://github.com/ivanpribela/testovid TestMyCode [194] WP Multi OS NetBeans https://github.com/testmycode/tmc-server CodeWorkout [74] WP C++, Java, Python, Ruby OS LTI https://github.com/web-cat/code-workout Arena [184] WP Imp/OO Git JavAssess (ASys) [115] LB Java OS http://personales.upv.es/josilga/ASys CodeOcean [226] WP Multi OS LTI https://github.com/openHPI/codeocean Jutge.org [176] WP Multi OS https://github.com/jutge-org GradeIT (PrUTOR) [62,173] CD C A https://prutor.ai neoESPA [148,150] WP C/C++, Java, Python A http://neoespa.pusan.ac.kr Enki (Mooshak) [170] WP Multi OS LTI + Eclipse https://mooshak2.dcc.fc.up.pt ProgEdu [50,252] CD Java OS Git https://github.com/ProgEdu-Organization/ProgEdu Aristotle [2] ...…”

Section: Table 3 General Information About the Selected Toolsmentioning

confidence: 99%

“…1 Token Pattern Approach 2 Several variants that are more tolerant JR -Java relection; UT -unit testing; OC -output comparison Table 5. Security models for untrusted code execution used in the selected tools Tool / Reference Model External Web-CAT [75] JVM (Java) YES PASS [255] User permissions YES Kattis [77] User permissions YES ECSpooler [13] User permissions YES STAGE (CodeCover) [172] JVM YES JACK [30,136,227] JVM YES URI Online Judge [32] User permissions YES CodeAssessor [241] User permissions NO VPL [203] Jail process YES CloudCoder [108] JVM (Java) + Jail process YES CodeRunner [60] User permissions YES JPLAS [83][84][85] Blacklist + JVM NO LX [61] User permissions NO Testovid within Protus [242] JVM YES TestMyCode [194] VM YES CodeWorkout [74] JVM (Java) + User permissions NO Arena [184] Container YES JavAssess (ASys) [115] JVM NO CodeOcean [226] Container YES Jutge.org [176] Jail process YES GradeIT (PrUTOR) [62,173] Container YES neoESPA [148,150] User permissions NO Enki (Mooshak) [170] User permissions NO ProgEdu [50,252] Container YES Aristotle [2] User permissions NO Submitty [180] Container YES Codelex [40] Jail process YES Dante [70] User permissions YES Coderiu [28] Container YES Edgar [156] Container YES Table…”

Section: Table 3 General Information About the Selected Toolsmentioning

confidence: 99%

See 1 more Smart Citation

Automated Assessment in Computer Science Education: A State-of-the-Art Review

Paiva

Leal

Figueira

2022

ACM Trans. Comput. Educ.

110

View full text Add to dashboard Cite

Practical programming competencies are critical to the success in computer science education and go-to-market of fresh graduates. Acquiring the required level of skills is a long journey of discovery, trial and error, and optimization seeking through a broad range of programming activities that learners must perform themselves. It is not reasonable to consider that teachers could evaluate all attempts that the average learner should develop multiplied by the number of students enrolled in a course, much less in a timely, deeply, and fairly fashion. Unsurprisingly, exploring the formal structure of programs to automate the assessment of certain features has long been a hot topic among CS education practitioners. Assessing a program is considerably more complex than asserting its functional correctness, as the proliferation of tools and techniques in the literature over the past decades indicates. Program efficiency, behavior, readability, among many other features, assessed either statically or dynamically, are now also relevant for automatic evaluation. The outcome of an evaluation evolved from the primordial boolean values to information about errors and tips on how to advance, possibly taking into account similar solutions. This work surveys the state-of-the-art in the automated assessment of CS assignments, focusing on the supported types of exercises, security measures adopted, testing techniques used, type of feedback produced, and the information they offer the teacher to understand and optimize learning. A new era of automated assessment, capitalizing on static analysis techniques and containerization, has been identified. Furthermore, this review presents several other findings from the conducted review, discusses the current challenges of the field, and proposes some future research directions.

show abstract

ProgEdu4Web: An automated assessment tool for motivating the learning of web programming course

Duong,

Chen

2024

Comp Applic In Engineering

View full text Add to dashboard Cite

The expanding software industry has increased the demand for proficient programming professionals. However, more than technical expertise is needed for successful software development; code quality assurance and effective teamwork are also indispensable. Unfortunately, recent graduates often exhibit deficiencies in programming, code quality assurance, and teamwork skills, highlighting the need for improvements in these areas. In response to these challenges, this study presents ProgEdu4Web, an innovative Automated Programming Assessment System designed to evaluate code quality and teamwork proficiency in web programming. The system integrates automatic source code evaluation tools for static code analysis, ensuring timely feedback on coding quality issues. To assess the effectiveness of ProgEdu4Web, a questionnaire and hypothesis model was employed, surveying a cohort of 111 students enrolled in web programming classes. The study's findings demonstrate that the system significantly alleviated instructor workload while motivating students to enhance their web programming with good code quality and teamwork abilities.

show abstract

The Impact of Iterative Assessment System on Programming Learning Behavior

Cited by 8 publications

References 3 publications

Analysis of Learning Behavior in an Automated Programming Assessment Environment: A Code Quality Perspective

Analysis of Learning Behavior in an Automated Programming Assessment Environment: A Code Quality Perspective

Automated Assessment in Computer Science Education: A State-of-the-Art Review

ProgEdu4Web: An automated assessment tool for motivating the learning of web programming course

Contact Info

Product

Resources

About