Comparing Code Explanations Created by Students and Large Language Models

Leinonen, Juho; Denny, Paul; MacNeil, Stephen; Sarsa, Sami; Bernstein, Seth; Kim, Joanne; Tran, Andrew; Hellas, Arto

doi:10.1145/3587102.3588785

Cited by 66 publications

(14 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The remarkable generative capabilities of LLMs have spurred the proliferation of LLM-based applications across various domains in writing scenarios. In the domain of code-related content generation, several studies investigate the potential of LLMs in crafting code explanations by comparing LLM-generated content with that sourced from students [21] or previous approaches [17], and validate its accuracy and comprehensibility. Some endeavors have integrated these auto-generated code explanations into educational scenarios.…”

Section: Llms For Document Generationmentioning

confidence: 99%

Bioinformatic analysis for potential biological processes and key targets of heart failure-related stroke

Liu

Chen

Zhang

et al. 2021

J. Zhejiang Univ. Sci. B

View full text Add to dashboard Cite

This study aimed to uncover underlying mechanisms and promising intervention targets of heart failure (HF)-related stroke. HF-related dataset GSE42955 and stroke-related dataset GSE58294 were obtained from the Gene Expression Omnibus (GEO) database. Weighted gene co-expression network analysis (WGCNA) was conducted to identify key modules and hub genes. Gene Ontology (GO) and pathway enrichment analyses were performed on genes in the key modules. Genes in HF-and stroke-related key modules were intersected to obtain common genes for HF-related stroke, which were further intersected with hub genes of stroke-related key modules to obtain key genes in HF-related stroke. Key genes were functionally annotated through GO in the Reactome and Cytoscape databases. Finally, key genes were validated in these two datasets and other datasets. HF-and stroke-related datasets each identified two key modules. Functional enrichment analysis indicated that protein ubiquitination, Wnt signaling, and exosomes were involved in both HF-and stroke-related key modules. Additionally, ten hub genes were identified in stroke-related key modules and 155 genes were identified as common genes in HF-related stroke. OTU deubiquitinase with linear linkage specificity (OTULIN) and nuclear factor interleukin 3-regulated (NFIL3) were determined to be the key genes in HF-related stroke. Through functional annotation, OTULIN was involved in protein ubiquitination and Wnt signaling, and NFIL3 was involved in DNA binding and transcription. Importantly, OTULIN and NFIL3 were also validated to be differentially expressed in all HF and stroke groups. Protein ubiquitination, Wnt signaling, and exosomes were involved in HF-related stroke. OTULIN and NFIL3 may play a key role in HF-related stroke through regulating these processes, and thus serve as promising intervention targets.

show abstract

Section: Llms For Document Generationmentioning

confidence: 99%

Bioinformatic analysis for potential biological processes and key targets of heart failure-related stroke

Liu

Chen

Zhang

et al. 2021

J. Zhejiang Univ. Sci. B

View full text Add to dashboard Cite

show abstract

“…The study indicated that while most generated content is fresh and logical, oversight is necessary to ensure content quality before student delivery. Several studies have analyzed the LLM models' ability to generate code explanations, contrasting the quality of these explanations with those provided by students [26,31,38,42].…”

Section: Related Workmentioning

confidence: 99%

“…Educators have assessed the performance of these LLMs in generating precise solutions to questions posed to students both within and beyond the classroom setting [16-18, 20, 32, 35, 37, 39, 42]. Other studies have looked at using LLMs to generate fresh programming exercises and code explanations [26,31,38,42], provide suggestive fixes to code errors [27] and generate personalized feedback for students based on their code submissions [13].…”

Section: Introductionmentioning

confidence: 99%

“It's not like Jarvis, but it's pretty close!” - Examining ChatGPT's Usage among Undergraduate Students in Computer Science

Budhiraja,

Joshi,

Challa

et al. 2024

Proceedings of the 26th Australasian Computing Education Conference

View full text Add to dashboard Cite

Large language models (LLMs) such as ChatGPT and Google Bard have garnered significant attention in the academic community. Previous research has evaluated these LLMs for various applications such as generating programming exercises and solutions. However, these evaluations have predominantly been conducted by instructors and researchers, not considering the actual usage of LLMs by students. This study adopts a student-first approach to comprehensively understand how undergraduate computer science students utilize ChatGPT, a popular LLM, released by OpenAI. We employ a combination of student surveys and interviews to obtain valuable insights into the benefits, challenges, and suggested improvements related to ChatGPT. Our findings suggest that a majority of students (over 57%) have a convincingly positive outlook towards adopting ChatGPT as an aid in coursework-related tasks. However, our research also highlights various challenges that must be resolved for long-term acceptance of ChatGPT amongst students. The findings from this investigation have broader implications and may be applicable to other LLMs and their role in computing education. CCS CONCEPTS• Human-centered computing → User studies; • Social and professional topics → Computer science education; • Computing methodologies → Artificial intelligence.

show abstract

“…Leveraging LLMs, educators can develop dynamic lesson plans with interactive elements, thereby captivating students' attention and fostering active participation (MacNeil et al, 2022). Moreover, LLMs can act as virtual teaching assistants, providing instant feedback on assignments and performance assessments (Leinonen et al, 2023). This enables educators to identify students' strengths and weaknesses more efficiently, leading to personalized interventions that enhance learning outcomes.…”

Section: Large Language Model In Research and Educationmentioning

confidence: 99%

Unraveling the landscape of large language models: a systematic review and future perspectives

Ding,

Wang

et al. 2023

JEBDE

View full text Add to dashboard Cite

PurposeThe rapid rise of large language models (LLMs) has propelled them to the forefront of applications in natural language processing (NLP). This paper aims to present a comprehensive examination of the research landscape in LLMs, providing an overview of the prevailing themes and topics within this dynamic domain.Design/methodology/approachDrawing from an extensive corpus of 198 records published between 1996 to 2023 from the relevant academic database encompassing journal articles, books, book chapters, conference papers and selected working papers, this study delves deep into the multifaceted world of LLM research. In this study, the authors employed the BERTopic algorithm, a recent advancement in topic modeling, to conduct a comprehensive analysis of the data after it had been meticulously cleaned and preprocessed. BERTopic leverages the power of transformer-based language models like bidirectional encoder representations from transformers (BERT) to generate more meaningful and coherent topics. This approach facilitates the identification of hidden patterns within the data, enabling authors to uncover valuable insights that might otherwise have remained obscure. The analysis revealed four distinct clusters of topics in LLM research: “language and NLP”, “education and teaching”, “clinical and medical applications” and “speech and recognition techniques”. Each cluster embodies a unique aspect of LLM application and showcases the breadth of possibilities that LLM technology has to offer. In addition to presenting the research findings, this paper identifies key challenges and opportunities in the realm of LLMs. It underscores the necessity for further investigation in specific areas, including the paramount importance of addressing potential biases, transparency and explainability, data privacy and security, and responsible deployment of LLM technology.FindingsThe analysis revealed four distinct clusters of topics in LLM research: “language and NLP”, “education and teaching”, “clinical and medical applications” and “speech and recognition techniques”. Each cluster embodies a unique aspect of LLM application and showcases the breadth of possibilities that LLM technology has to offer. In addition to presenting the research findings, this paper identifies key challenges and opportunities in the realm of LLMs. It underscores the necessity for further investigation in specific areas, including the paramount importance of addressing potential biases, transparency and explainability, data privacy and security, and responsible deployment of LLM technology.Practical implicationsThis classification offers practical guidance for researchers, developers, educators, and policymakers to focus efforts and resources. The study underscores the importance of addressing challenges in LLMs, including potential biases, transparency, data privacy, and responsible deployment. Policymakers can utilize this information to shape regulations, while developers can tailor technology development based on the diverse applications identified. The findings also emphasize the need for interdisciplinary collaboration and highlight ethical considerations, providing a roadmap for navigating the complex landscape of LLM research and applications.Originality/valueThis study stands out as the first to examine the evolution of LLMs across such a long time frame and across such diversified disciplines. It provides a unique perspective on the key areas of LLM research, highlighting the breadth and depth of LLM’s evolution.

show abstract

Comparing Code Explanations Created by Students and Large Language Models

Cited by 66 publications

References 33 publications

Bioinformatic analysis for potential biological processes and key targets of heart failure-related stroke

Bioinformatic analysis for potential biological processes and key targets of heart failure-related stroke

“It's not like Jarvis, but it's pretty close!” - Examining ChatGPT's Usage among Undergraduate Students in Computer Science

Unraveling the landscape of large language models: a systematic review and future perspectives

Contact Info

Product

Resources

About