No More Pencils No More Books: Capabilities of Generative AI on Irish and UK Computer Science School Leaving Examinations

Mahon, Joyce; Mac Namee, Brian; Becker, Brett A.

doi:10.1145/3610969.3610982

Cited by 1 publication

(1 citation statement)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These tools can write, explain, and debug code in ways that simply were not possible just two years ago [19]. Researchers have been quick to measure the ability of these LLMs with regard to typical computer science programs [7,8], code explanations [12], exams [17], and even Parsons Problems (mixed-up code problems) [24]. This has led to instructors having many concerns, such as students using autogenerated code that they don't understand [23] or using these tools to do their work for them [1,11].…”

Section: Introductionmentioning

confidence: 99%

Enhancing Programming Error Messages in Real Time with Generative AI

Kimmel,

Geisert,

Yaro

et al. 2024

Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Generative AI is changing the way that many disciplines are taught, including computer science. Researchers have shown that generative AI tools are capable of solving programming problems, writing extensive blocks of code, and explaining complex code in simple terms. Particular promise has been shown in using generative AI to enhance programming error messages. Both students and instructors have complained for decades that these messages are often cryptic and difficult to understand. Yet recent work has shown that students make fewer repeated errors when enhanced via GPT-4. We extend this work by implementing feedback from ChatGPT for all programs submitted to our automated assessment tool, Athene, providing help for compiler, run-time, and logic errors. Our results indicate that adding generative AI to an automated assessment tool does not necessarily make it better and that design of the interface matters greatly to the usability of the feedback that GPT-4 provided.

show abstract