A Comparative Study of Free Self-Explanations and Socratic Tutoring Explanations for Source Code Comprehension

Tamang, Lasang Jimba; Alshaikh, Zeyad; Khayi, Nisrine Ait; Oli, Priti; Rus, Vasile

doi:10.1145/3408877.3432423

Cited by 13 publications

(5 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This could involve the AI assistant highlighting critical sections of the code and prompting students to answer comprehension questions about these parts. Additionally, future AI tools could foster independent problem-solving using the Socratic Method [57] and transforming code generation requests into a series of problemsolving questions. With each correct response, the tool would reveal the code segment corresponding to that question.…”

Section: D3: Balancing the Directness Of Ai Responsesmentioning

confidence: 99%

CodeAid: Evaluating a Classroom Deployment of an LLM-based Programming Assistant that Balances Student and Educator Needs

Kazemitabaar,

Ye,

Wang

et al. 2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Timely, personalized feedback is essential for students learning programming. LLM-powered tools like ChatGPT offer instant support, but reveal direct answers with code, which may hinder deep conceptual engagement. We developed CodeAid, an LLM-powered programming assistant delivering helpful, technically correct responses, without revealing code solutions. CodeAid answers conceptual questions, generates pseudo-code with line-by-line explanations, and annotates student's incorrect code with fix suggestions. We deployed CodeAid in a programming class of 700 students for a 12-week semester. A thematic analysis of 8,000 usages of CodeAid was performed, further enriched by weekly surveys, and 22 student interviews. We then interviewed eight programming educators to gain further insights. Our findings reveal four design considerations for future educational AI assistants: D1) exploiting AI's unique benefits; D2) simplifying query formulation while promoting cognitive engagement; D3) avoiding direct responses while encouraging motivated learning; and D4) maintaining transparency and control for students to asses and steer AI responses.

show abstract

Section: D3: Balancing the Directness Of Ai Responsesmentioning

confidence: 99%

CodeAid: Evaluating a Classroom Deployment of an LLM-based Programming Assistant that Balances Student and Educator Needs

Kazemitabaar,

Ye,

Wang

et al. 2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

show abstract

“…Wang, Rao, Zhi, Marwan, Gao, & Price, 2020;Zhi, Price, Marwan, Milliken, Barnes, & Chi, 2019). There are a number of computer tutors that offer guidance for writing code (Bau, Gray, Kelleher, Sheldon, & Turbak, 2017;Costa & Miranda, 2017;Weintrop & Wilensky, 2018;Whittall, Prashandi, Himasha, De Silva, & Suriyawansa, 2017), but to date there is little work on using tutoring systems to support code tracing, with some notable exceptions (Caughey & Muldner, 2023;Chiarelli, Markova, & Muldner, 2023;Huang, Brusilovsky, Guerra, Koedinger, & Schunn, 2023;Kumar, 2020;Nelson, Xie, & Ko, 2017;Tamang, Alshaikh, Khayi, Oli, & Rus, 2021). These exceptions are described in the subsequent chapter, but here I note that some of these tutors are designed for a traditional computer science audience and so may use instructions that are too technical for other populations.…”

Section: Overviewmentioning

confidence: 99%

“…I contribute to the understanding of how to effectively design assistance in both example and problem contexts. Research in this area is needed as currently there is only a handful of tutors for code tracing (Caughey & Muldner, 2023;Chiarelli et al, 2023;Huang et al, 2023;Kumar, 2014;Nelson et al, 2017;Tamang et al, 2021). The CT-Tutor is currently the only one to provide dynamic examples and problems that scaffold code tracing through the interface design.…”

Section: Contributionsmentioning

confidence: 99%

“…3. Several code-tracing tutors have included self-explanation prompts in an example context (Kumar, 2014;Nelson et al, 2017) and in a problem context (Caughey & Muldner, 2023;Tamang et al, 2021). However, there is no work investigating how students self-explain in a code-tracing tutoring system in the absence of prompts.…”

Section: Contributionsmentioning

confidence: 99%

“…The authors speculated that this latter unexpected finding was due to the higher cognitive load that translation activities required, which may have interfered with the learning of code tracing. Tamang et al (2021) also evaluated a tutoring system that scaffolded code-tracing problems with self-explanation prompts. There were three conditions: self-explanation, Socratic method, and a control condition.…”

Section: Tutoring Systems For Programming Activitiesmentioning

confidence: 99%

See 2 more Smart Citations

Investigating the Effect of Interface Scaffolding and Example Design on Learning and Performance in a Code-Tracing Tutor

Jennings

View full text Add to dashboard Cite

A foundational programming skill is code tracing. It involves simulating how a computer executes programs by tracking variable values and flow of program execution. Learning this skill is challenging and so research is needed to identify effective forms of assistance. This dissertation involves the design and evaluation of a computer tutor, called CT-Tutor.The tutor provided assistance for code tracing through interface scaffolding during code tracing as well as worked examples. The tutor was evaluated in three studies. Study 1 involved four versions of CT-Tutor to evaluate the effect of two levels of instructional scaffolding (high/reduced) and two types of instructional order (example-first/problemfirst). Contrary to my hypothesis, there was no evidence for a learning benefit from high scaffolding. However, the high-scaffolding version improved performance by reducing the number of attempts needed to get an answer correct. Instructional order also influenced performance. The example-first group spent more time per example, less time per problem, and required fewer attempts to produce correct answers.To shed light on student reasoning with CT-Tutor, Study 2 used a think-aloud protocol to analyze students' self-explanations and reading behaviors with the high-and reduced-scaffolding versions of the CT-Tutor; an exit interview was used to obtain data on the tutor's usability. Self-explanation was the main variable of interest, as prior research demonstrated it is beneficial for learning. There was no significant difference in the number of self-explanation between the two scaffolding versions. The exit interviews revealed that the CT-Tutor example design could be improved. i The CT-Tutor was subsequently re-designed to provide either dynamic and static examples. Previous research suggested that domains involving change over time are best presented by dynamic examples; code tracing is one such domain. The effect of example type on learning was not significant. The dynamic-example group spent more time on the examples, had marginally fewer incorrect attempts, and spent less time producing incorrect answers. In sum, ignoring condition, the CT-Tutor improved learning. However, the effect of the experimental manipulations was inconclusive. Performance was significantly affected by the experimental manipulations: the high-scaffolding problem interface, examplefirst instructional order, and dynamic examples improved code-tracing performance. David for your invaluable advice, guidance, and feedback which have been instrumental in shaping this dissertation. I extend my appreciation to my examining committee: my internal examiner, Robert Biddle, and my external examiner, Gord McCalla. Your insightful comments and constructive feedback have significantly enriched my work. This dissertation would not have been possible without my family. My parents, who have always emphasized the importance of education and believed in me throughout the highs and lows of this degree; and my sister, whose reactions to the magnitude of work inv...

show abstract

Automatic Question Generation for Scaffolding Self-explanations for Code Comprehension

Tamang

Banjade

Chapagain

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

A Comparative Study of Free Self-Explanations and Socratic Tutoring Explanations for Source Code Comprehension

Cited by 13 publications

References 27 publications

CodeAid: Evaluating a Classroom Deployment of an LLM-based Programming Assistant that Balances Student and Educator Needs

CodeAid: Evaluating a Classroom Deployment of an LLM-based Programming Assistant that Balances Student and Educator Needs

Investigating the Effect of Interface Scaffolding and Example Design on Learning and Performance in a Code-Tracing Tutor

Automatic Question Generation for Scaffolding Self-explanations for Code Comprehension

Contact Info

Product

Resources

About