Towards understanding the effective design of automated formative feedback for programming assignments

Hao, Qiang; Smith, David H.; Ding, Li; Ko, Amy J.; Ottaway, Camille; Wilson, Jack; Arakawa, Kai; Turcan, Alistair; Poehlman, Timothy; Greer, Tyler

doi:10.1080/08993408.2020.1860408

Cited by 28 publications

(12 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our approach aims at fostering self-comprehension of studentauthored solutions by means of the QLCs. We decided to provide a feedback type of knowledge of correct response (KCR [13]), which revealed a positive effect in a programming learning context when compared to knowledge of result (KR) without informing the correct answer [3].…”

Section: Related Workmentioning

confidence: 99%

Jask

Santos

Soares

Garrido

et al. 2022

Proceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 1

View full text Add to dashboard Cite

We present Jask, a system capable of generating questions about a learner's code written in Java. Given Java code as input, Jask provides a set of meaningful questions formulated in terms of the actual code (using its constructs and identifiers) and the corresponding correct answers. We integrated Jask in a web-based system where students submit their code (e.g., from lab exercises), answer questions about it, and obtain immediate formative feedback with the correct answers. An initial study involving 123 distinct introductory programming students providing 2274 answers revealed that questions pertaining to program dynamics tend to register low scores, possibly evidencing fragile comprehension of programming constructs. Participants were surveyed, revealing a positive view towards the usefulness of Jask, especially with respect to consolidating terminology.

show abstract

Section: Related Workmentioning

confidence: 99%

Jask

Santos

Soares

Garrido

et al. 2022

Proceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 1

View full text Add to dashboard Cite

show abstract

“…a proof rule cannot be applied. Recent research indicates that this kind of feedback impacts learning in computer science significantly, and is sufficient to allow students to move forward in most cases [18].…”

Section: Discussion and Future Workmentioning

confidence: 99%

Using Isabelle in Two Courses on Logic and Automated Reasoning

Villadsen¹,

Jacobsen²

2021

Formal Methods Teaching

View full text Add to dashboard Cite

We present our experiences teaching two courses on formal methods and detail the contents of the courses and their positioning in the curriculum. The first course is a bachelor course on logical systems and logic programming, with a focus on classical first-order logic and automatic theorem proving. The second course is a master course on automated reasoning, with a focus on classical higher-order logic and interactive theorem proving. The proof assistant Isabelle is used in both courses, using Isabelle/Pure as well as Isabelle/HOL. We describe our online tools to be used with Isabelle/HOL, in particular the Sequent Calculus Verifier (SeCaV) and the Natural Deduction Assistant (NaDeA). We also describe our innovative Students' Proof Assistant which is formally verified in Isabelle/HOL and integrated in Isabelle/jEdit using Isabelle/ML.

show abstract

“…As noted by Hao et al (2022), hundreds of studies have adopted the above feedback classification inclusive of KR, KCR, and EF.…”

Section: Feedback Typesmentioning

confidence: 99%

Where less is more: Limited feedback in formative online multiple‐choice tests improves student self‐regulation

Say,

Visentin,

Saunders

et al. 2023

Computer Assisted Learning

View full text Add to dashboard Cite

BackgroundFormative online multiple‐choice tests are ubiquitous in higher education and potentially powerful learning tools. However, commonly used feedback approaches in online multiple‐choice tests can discourage meaningful engagement and enable strategies, such as trial‐and‐error, that circumvent intended learning outcomes. These strategies will not prepare graduates as self‐regulated learners, nor for the complexities of contemporary work settings.ObjectivesTo investigate whether providing only a score after formative online multiple‐choice test attempts (score‐only feedback) increases the likelihood of students to engage in self‐regulated learning compared with more directive feedback. Measurable outcomes included deeper learning, collaboration, information seeking, and satisfaction.MethodsData in this mixed methods study were collected from nursing students through surveys, test results, focus groups, and student discussion board contributions. A quasi‐experimental design was used for quantitative data, and qualitative data were analysed thematically against domains of self‐regulated learning.Results and ConclusionsStudents receiving score‐only feedback were more cognitively engaged with the content, collaborated constructively, and sought out richer sources of information. However, it was also associated with lower satisfaction. In this study, minimal feedback created states of uncertainty, which resulted in the activation of self‐regulatory actions.Implications for PracticeProviding overly directive feedback for formative online multiple‐choice tests is conducive to surface‐level learning strategies. By minimising feedback and allowing for extended states of uncertainty, students are more likely to regulate their learning through self‐assessment and problem‐solving strategies, all of which are required by graduates to meet the challenges of real‐world work settings.

show abstract

Towards understanding the effective design of automated formative feedback for programming assignments

Cited by 28 publications

References 65 publications

Jask

Jask

Using Isabelle in Two Courses on Logic and Automated Reasoning

Where less is more: Limited feedback in formative online multiple‐choice tests improves student self‐regulation

Contact Info

Product

Resources

About