Automatic Code Documentation Generation Using GPT-3

Khan, Junaed Younus; Uddin, Gias

doi:10.1145/3551349.3559548

Cited by 40 publications

(9 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This was sufficient for our study: an explanation failure occurred only 13 times out of 159 queries in the grounded condition, on average 1.08 times per user. We do not claim that our algorithm is the most effective, and future work could explore alternative ways of producing grounded utterances, such as leveraging the LLM itself [45,72]. Our system is limited in assuming a single well-defined relational data table.…”

Section: Limitationsmentioning

confidence: 97%

“What It Wants Me To Say”: Bridging the Abstraction Gap Between End-User Programmers and Code-Generating Large Language Models

Liu¹,

Sarkar²,

Negreanu³

et al. 2023

Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Code-generating large language models map natural language to code. However, only a small portion of the infinite space of naturalistic utterances is effective at guiding code generation. For non-expert end-user programmers, learning this is the challenge of abstraction matching. We examine this challenge in the specific context of data analysis in spreadsheets, in a system that maps the user's natural language query to Python code using the Codex generator, executes the code, and shows the result. We propose grounded abstraction matching, which bridges the abstraction gap by translating the code back into a systematic and predictable naturalistic utterance. In a between-subjects, think-aloud study (n=24), we compare grounded abstraction matching to an ungrounded alternative based on previously established query framing principles. We find that the grounded approach improves end-users' understanding of the scope and capabilities of the code-generating model, and the kind of language needed to use it effectively. CCS CONCEPTS• Human-centered computing → Natural language interfaces; Interactive systems and tools; Empirical studies in HCI .

show abstract

Section: Limitationsmentioning

confidence: 97%

“What It Wants Me To Say”: Bridging the Abstraction Gap Between End-User Programmers and Code-Generating Large Language Models

Liu¹,

Sarkar²,

Negreanu³

et al. 2023

Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

show abstract

“…Recently, a new generation of LLMs demonstrated the capability to understand and generate both natural languages and computer languages. GPT-3 was examined in writing code explanations [52], documentation [44], and providing feedback for assignments [3]. Soon, educators started to believe that Codex could be used to solve simple programming problems [27,90].…”

Section: Related Work 21 Llms For Computational Programming and Modelingmentioning

confidence: 99%

Learning Agent-based Modeling with LLM Companions: Experiences of Novices and Experts Using ChatGPT & NetLogo Chat

Chen,

Lu,

et al. 2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Large Language Models (LLMs) have the potential to fundamentally change the way people engage in computer programming. Agentbased modeling (ABM) has become ubiquitous in natural and social sciences and education, yet no prior studies have explored the potential of LLMs to assist it. We designed NetLogo Chat to support the learning and practice of NetLogo, a programming language for ABM. To understand how users perceive, use, and need LLM-based interfaces, we interviewed 30 participants from global academia, industry, and graduate schools. Experts reported more perceived benefits than novices and were more inclined to adopt LLMs in their workflow. We found significant differences between experts and novices in their perceptions, behaviors, and needs for human-AI collaboration. We surfaced a knowledge gap between experts and novices as a possible reason for the benefit gap. We identified guidance, personalization, and integration as major needs for LLMbased interfaces to support the programming of ABM.

show abstract

“…The research community explored GLLMs for coding tasks across various languages like Java [251], [252], [255], [260], [263], [264], [266], [267], [269], [270], Python [253], [254], [256]- [258], [260], [262], [263], [265], [267], [268], [271], PHP [260], GO [260], Ruby [260], JavaScript [260], C [261], [268], C++ [259], [268], Julia [268], and MATLAB [268]. Most of the research works focused on Python and Java languages, while a few research works focused on other languages like GO, PHP, GO, Ruby, JavaScript, C, C++, Julia and MATLAB.…”

Section: Research Work Exploring Gllms For Various Coding Tasksmentioning

confidence: 99%

“…In all the above discussed research works, the performance of GLLMs in various coding tasks is promising but still lags behind SOTA results. Some of the research works [251], [254], [260], [262], [266] demonstrated that GLLMs can achieve SOTA results in coding tasks. Xia et al [251] proposed ChatRepair, an automatic program repair tool based on ChatGPT.…”

Section: Research Work Exploring Gllms For Various Coding Tasksmentioning

confidence: 99%

“…It successfully resolves 114 and 48 bugs on Defects4j 1.2 and 2.0 [278], respectively, outperforming the previous best by 15 and 17 bugs, respectively. Khan et al [260] explored Codex, GPT-3 family model pretrained on natural and programming languages to automate code documentation generation. The evaluation results on six programming languages showed that Codex, with just one example, outperforms existing approaches by a large margin of 11.2%.…”

Section: Research Work Exploring Gllms For Various Coding Tasksmentioning

confidence: 99%

See 1 more Smart Citation