Novice programmers' faults &amp; failures in GOAL programs: Empirical observations and lessons

Winikoff, Michael

doi:10.26686/wgtn.12909977.v1

Cited by 2 publications

(3 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The paper "Slicing Agent Programs for More Efficient Verification" by Michael Winikoff, Louise A. Dennis and Michael Fisher focuses on formal verification of agent programs using model checking. Formal verification of cognitive agents is highly desirable, since the complexity of their behavior makes assurance via traditional software testing infeasible [49,50]. However, current state-of-the-art techniques and tools for model checking cognitive agent programs are not able to deal with larger programs.…”

Section: Formal Analysis and Techniquesmentioning

confidence: 99%

“…Programming plans or rules that are applicable in all situations is very hard. Plan contexts typically used to control applicability are a key source of bugs in agent programming [50] and the need to construct contexts to cover all eventualities results in programs containing a proliferation of plans and a concomitant reduction in the transparency of code -contrary to an off-stated assumption that agent programming languages encourage a declarative and understandable programming style. The issue typically manifests as control over the selection of plans, but is also relevant to the selection of goals and the selection and scheduling of intentions (two areas that have received less attention from the community though they have not been entirely ignored, see for instance [34,43,44,52]).…”

Section: What Are the Challenges?mentioning

confidence: 99%

See 1 more Smart Citation

Engineering Multi-Agent Systems

Mascardi

Weyns

Ricci

et al. 2019

SIGSOFT Softw. Eng. Notes

Self Cite

View full text Add to dashboard Cite

The continuous integration of software-intensive systems together with the ever-increasing computing power offer a breeding ground for intelligent agents and multi-agent systems (MAS) more than ever before. Over the past two decades, a wide variety of languages, models, techniques and methodologies have been proposed to engineer agents and MAS. Despite this substantial body of knowledge and expertise, the systematic engineering of large-scale and open MAS still poses many challenges. Researchers and engineers still face fundamental questions regarding theories, architectures, languages, processes, and platforms for designing, implementing, running, maintaining, and evolving MAS. This paper reports on the results of the 6th International Workshop on Engineering Multi-Agent Systems (EMAS 2018, 14th-15th of July, 2018, Stockholm, Sweden), where participants discussed the issues above focusing on the state of affairs and the road ahead for researchers and engineers in this area.

show abstract

Section: Formal Analysis and Techniquesmentioning

confidence: 99%

Section: What Are the Challenges?mentioning

confidence: 99%

Engineering Multi-Agent Systems

Mascardi

Weyns

Ricci

et al. 2019

SIGSOFT Softw. Eng. Notes

Self Cite

View full text Add to dashboard Cite

show abstract

“…These mistakes pose substantial hurdles to students as they strive to develop their programming skills. Despite extensive efforts by computing education researchers and practitioners to establish taxonomies and recognize patterns of common programming errors [1,6,36,37,60] , the process of effectively detecting and resolving bugs remains a persistent challenge.…”

Section: Introductionmentioning

confidence: 99%

The Implications of Large Language Models for CS Teachers and Students

MacNeil

Kim

Leinonen

et al. 2023

Proceedings of the 54th ACM Technical Symposium on Computer Science Education v. 2

View full text Add to dashboard Cite

Identifying and resolving logic errors can be one of the most frustrating challenges for novices programmers. Unlike syntax errors, for which a compiler or interpreter can issue a message, logic errors can be subtle. In certain conditions, buggy code may even exhibit correct behavior -in other cases, the issue might be about how a problem statement has been interpreted. Such errors can be hard to spot when reading the code, and they can also at times be missed by automated tests. There is great educational potential in automatically detecting logic errors, especially when paired with suitable feedback for novices. Large language models (LLMs) have recently demonstrated surprising performance for a range of computing tasks, including generating and explaining code. These capabilities are closely linked to code syntax, which aligns with the next token prediction behavior of LLMs. On the other hand, logic errors relate to the runtime performance of code and thus may not be as well suited to analysis by LLMs. To explore this, we investigate the performance of two popular LLMs, GPT-3 and GPT-4, for detecting and providing a novice-friendly explanation of logic errors. We compare LLM performance with a large cohort of introductory computing students (𝑛 = 964) solving the same error detection task. Through a mixed-methods analysis of student and model responses, we observe significant improvement in logic error identification between the previous and current generation of LLMs, and find that both LLM generations significantly outperform students.We outline how such models could be integrated into computing education tools, and discuss their potential for supporting students when learning programming.

show abstract

Novice programmers' faults & failures in GOAL programs: Empirical observations and lessons

Cited by 2 publications

References 5 publications

Engineering Multi-Agent Systems

Engineering Multi-Agent Systems

The Implications of Large Language Models for CS Teachers and Students

Contact Info

Product

Resources

About