Automated Measurement of Competencies and Generation of Feedback in Object-Oriented Programming Courses

Krugel, Johannes; Hubwieser, Peter; Goedicke, Michael; Striewe, Michael; Talbot, Mike; Olbricht, Christoph J.; Schypula, Melanie; Zettler, Simon

doi:10.1109/educon45650.2020.9125323

Cited by 15 publications

(10 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These include 60 publications that either focused on the same tool, were missing the description of most of the features we aimed to capture, or were considered irrelevant as no further link to them could be found online, i.e., they were only a prototype. From the remaining 68, we have selected 10 for their high and recent development activity -Submitty [180], Edgar [156], CodeOcean [226], ProgEdu [50], CodeRunner [60], Aristotle [2], Jutge.org [176], CodeWorkout [74], TestMyCode [194], and VPL [203] -, 5 for being active for a long period of time -Kattis [77], JACK [30,136], Mooshak [170], URI Online Judge [32], and Web-CAT [75] -, and 15 picked in a random sampling step.…”

Section: Toolsmentioning

confidence: 99%

Automated Assessment in Computer Science Education: A State-of-the-Art Review

Paiva

Leal

Figueira

2022

ACM Trans. Comput. Educ.

106

View full text Add to dashboard Cite

Practical programming competencies are critical to the success in computer science education and go-to-market of fresh graduates. Acquiring the required level of skills is a long journey of discovery, trial and error, and optimization seeking through a broad range of programming activities that learners must perform themselves. It is not reasonable to consider that teachers could evaluate all attempts that the average learner should develop multiplied by the number of students enrolled in a course, much less in a timely, deeply, and fairly fashion. Unsurprisingly, exploring the formal structure of programs to automate the assessment of certain features has long been a hot topic among CS education practitioners. Assessing a program is considerably more complex than asserting its functional correctness, as the proliferation of tools and techniques in the literature over the past decades indicates. Program efficiency, behavior, readability, among many other features, assessed either statically or dynamically, are now also relevant for automatic evaluation. The outcome of an evaluation evolved from the primordial boolean values to information about errors and tips on how to advance, possibly taking into account similar solutions. This work surveys the state-of-the-art in the automated assessment of CS assignments, focusing on the supported types of exercises, security measures adopted, testing techniques used, type of feedback produced, and the information they offer the teacher to understand and optimize learning. A new era of automated assessment, capitalizing on static analysis techniques and containerization, has been identified. Furthermore, this review presents several other findings from the conducted review, discusses the current challenges of the field, and proposes some future research directions.

show abstract

Section: Toolsmentioning

confidence: 99%

Automated Assessment in Computer Science Education: A State-of-the-Art Review

Paiva

Leal

Figueira

2022

ACM Trans. Comput. Educ.

106

View full text Add to dashboard Cite

show abstract

“…Zeller states that the programming style improved faster than in earlier courses, which was seemingly a result of the peer review of the code by students [19]. While most of the systems described in [12]- [21] have been applied to university courses, MOOCs have also been using automatic and semi-automatic review systems [20], [21], [23], [24]. Systems like the JACK system by Krugel et al [20], [23], ArTEMiS [24], as well as the INGInious platform described by [25] showcase the usage of these systems in a distributed online course format.…”

Section: Related Workmentioning

confidence: 99%

Transitioning to a Large-Scale Distributed Programming Course

Schmiedmayer

Reimer

Jovanović

et al. 2020

2020 IEEE 32nd Conference on Software Engineering Education and Training (CSEE&T)

View full text Add to dashboard Cite

hall with a capacity of about 100 seats. Students could ask for support by simply raising their hand, and tutors were standing by to help. Every participating student brought their own private computer to the lecture hall or was provided with a computer with a preinstalled Xcode integrated development environment (IDE) and tools to take part in the programming exercises and lectures. In general, the programming course is scheduled to run for ten days over a two-week period before the beginning of each semester. During the course, students learn the fundamentals of the Swift programming language and how to develop iOS and iPadOS applications. As Swift, iOS, and iPadOS development concepts change regularly, the curriculum of the course is adapted every semester. Due to the university lockdown starting four weeks prior to the scheduled start of the programming course, we were confronted with additional challenges as we were unable to conduct the course in a personalized, on-site setting. In this paper, we describe the experiences, challenges, methods, and outcomes of transitioning the Swift programming course to a large-scale distributed format. II. TEACHING CONCEPTS In the following paragraphs, we provide a summary of the teaching concepts relating to the Swift programming course and the subsequent multi-project capstone course. A. Capstone Course: iPraktikum The iPraktikum is a practical course which allows students to experience software engineering first-hand in a projectbased structure [1]. It focuses on mobile applications in larger system architectures that can include, for example, IoT devices or server components. With up to 100 students each semester, the course is split into a maximum of 12 teams to develop applications for customers from industry. Each team consists of one project leader (a doctoral candidate or post-doc from the host chair), one coach (a student who has already participated in the capstone course before), and the student developers. The iPraktikum is offered to undergraduate and graduate students from different fields of study such as biomedical computing, computer science, data engineering and analytics, electrical engineering, information systems, management and technology, mathematics, mechanical engineering, physics,

show abstract

“…Other research aiming at investigating the difficulty of content in computing education analyzing how students learn to program is mostly related to higher education [35], other block-based languages, such as Scratch [36][37] [38][39][40] [41], LaPlaya [42], and SNAP! based environments [43], object-oriented programming [44], etc.…”

Section: Introductionmentioning

confidence: 99%

An Item Response Theory Analysis of Algorithms and Programming Concepts in App Inventor Projects

Alves

Wangenheim

Hauck

et al. 2021

Anais Do I Simpósio Brasileiro De Educação Em Computação (EDUCOMP 2021)

View full text Add to dashboard Cite

Computing education is often introduced in K-12 focusing on algorithms and programming concepts using block-based programming environments, such as App Inventor. Yet, learning programming is a complex process and novices struggle with several difficulties. Thus, to be effective, instructional units need to be designed regarding not only the content but also its sequencing taking into consideration difficulties related to the concepts and the idiosyncrasies of programming environments. Such systematic sequencing can be based on large-scale project analyses by regarding the volition, incentive, and opportunity of students to apply the relevant program constructs as latent psychometric constructs using Item Response Theory to obtain quantitative ?difficulty? estimates for each concept. Therefore, this article presents the results of a large-scale data-driven analysis of the demonstrated use in practice of algorithms and programming concepts in App Inventor. Based on a dataset of more than 88,000 App Inventor projects assessed automatically with the ANON rubric, we perform an analysis using Item Response Theory. The results demonstrate that the easiness of some concepts can be explained by their inherent characteristics, but also due to the characteristics of App Inventor as a programming environment. These results can help teachers, instructional and curriculum designers in the sequencing, scaffolding and assessment design of programming education in K-12.

show abstract

Automated Measurement of Competencies and Generation of Feedback in Object-Oriented Programming Courses

Cited by 15 publications

References 28 publications

Automated Assessment in Computer Science Education: A State-of-the-Art Review

Automated Assessment in Computer Science Education: A State-of-the-Art Review

Transitioning to a Large-Scale Distributed Programming Course

An Item Response Theory Analysis of Algorithms and Programming Concepts in App Inventor Projects

Contact Info

Product

Resources

About