Repair Is Nearly Generation: Multilingual Program Repair with LLMs

Joshi, Harshit; Cambronero, José; Gulwani, Sumit; Le, Vu; Radiček, Ivan; Verbruggen, Gust

doi:10.48550/arxiv.2208.11640

Cited by 8 publications

(15 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Github Copilot uses GPT-3 for automated code generation from natural language inputs [8]. Several researchers have addressed code generation [8], [36], docstring generation [8], [60], and code repair [61], [62] problems. Bareiß et al [63] show how fewshot learning can be effective at (i) code mutation; (ii) test oracle generation from natural language documentation; and (iii) test case generation task.…”

Section: B Llms In Software Engineeringmentioning

confidence: 99%

Recommending Root-Cause and Mitigation Steps for Cloud Incidents using Large Language Models

Ahmed¹,

Ghosh²,

Bansal³

et al. 2023

Preprint

View full text Add to dashboard Cite

Incident management for cloud services is a complex process involving several steps and has a huge impact on both service health and developer productivity. On-call engineers require significant amount of domain knowledge and manual effort for root causing and mitigation of production incidents. Recent advances in artificial intelligence has resulted in state-ofthe-art large language models like GPT-3.x (both GPT-3.0 and GPT-3.5), which have been used to solve a variety of problems ranging from question answering to text summarization. In this work, we do the first large-scale study to evaluate the effectiveness of these models for helping engineers root cause and mitigate production incidents. We do a rigorous study at Microsoft, on more than 40,000 incidents and compare several large language models in zero-shot, fine-tuned and multi-task setting using semantic and lexical metrics. Lastly, our human evaluation with actual incident owners show the efficacy and future potential of using artificial intelligence for resolving cloud incidents.

show abstract

Section: B Llms In Software Engineeringmentioning

confidence: 99%

Recommending Root-Cause and Mitigation Steps for Cloud Incidents using Large Language Models

Ahmed¹,

Ghosh²,

Bansal³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Recent studies have shown that mining relevant bug-fix patterns from existing search space (Jiang et al, 2018) and external repair templates from StackOverflow (Liu and Zhong, 2018) can significantly benefit APR models. Joshi et al (2022) intuitively rank a collection of bugfix pairs based on the similarity of error messages to develop few-shot prompts. They incorporate compiler error messages into a large programming language model Codex (Chen et al, 2021) for multilingual APR.…”

Section: Codementioning

confidence: 99%

Retrieving Multimodal Information for Augmented Generation: A Survey

Zhao¹,

Chen²,

Wang³

et al. 2023

Preprint

View full text Add to dashboard Cite

In this survey, we review methods that retrieve multimodal knowledge to assist and augment generative models. This group of works focuses on retrieving grounding contexts from external sources, including images, codes, tables, graphs, and audio. As multimodal learning and generative AI have become more and more impactful, such retrieval augmentation offers a promising solution to important concerns such as factuality, reasoning, interpretability, and robustness. We provide an in-depth review of retrieval-augmented generation in different modalities and discuss potential future directions. As this is an emerging field, we continue to add new papers and methods.

show abstract

“…We then compare PCR-Chain to state-of-the-art (SOTA) methods such as BIFI [39], CURE [27] and RING [40], in both dynamically-typed (Python) and statically-typed (Java) programming languages. In Python, PCR-Chain outperforms SOTA LLM-based methods such as BIFI by 13.7% accuracy and ICL-based methods such as RING by 5%.…”

Section: Step5 Error Message Enhancementioning

confidence: 99%

“…PCR-Chain-D For Python, we compare PCR-Chain with BIFI [39], an LLM-based SOTA method, and RING [40], an ICL-based SOTA method. In Java, we compare PCR-Chain with CURE, an LLM-based SOTA method, and RING.…”

Section: (A)mentioning

confidence: 99%

See 1 more Smart Citation

Prompt-tuned Code Language Model as a Neural Knowledge Base for Type Inference in Statically-Typed Partial Code

Huang

Yuan

Xing

et al. 2022

Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering

View full text Add to dashboard Cite

Control Flow Graphs (CFGs) are essential for visualizing, understanding and analyzing program behavior. For statically-typed programming language like Java, developers obtain CFGs by using bytecode-based methods for compilable code and Abstract Syntax Tree (AST)-based methods for partially uncompilable code. However, explicit syntax errors during AST construction and implicit semantic errors caused by bad coding practices can lead to behavioral loss and deviation of CFGs. To address the issue, we propose a novel approach that leverages the error-tolerant and understanding ability of pre-trained Large Language Models (LLMs) to generate CFGs. Our approach involves a Chain of Thought (CoT) with four steps: structure hierarchy extraction, nested code block extraction, CFG generation of nested code blocks, and fusion of all nested code blocks' CFGs. To address the limitations of the original CoT's single-prompt approach (i.e., completing all steps in a single generative pass), which can result in an "epic" prompt with hard-to-control behavior and error accumulation, we break down the CoT into an AI chain with explicit sub-steps. Each sub-step corresponds to a separate AI-unit, with an effective prompt assigned to each unit for interacting with LLMs to accomplish a specific purpose. Our experiments confirmed that our method outperforms existing CFG tools in terms of node and edge coverage, especially for incomplete or erroneous code. We also conducted an ablation experiment and confirmed the effectiveness of AI chain design principles: Hierarchical Task Breakdown, Unit Composition, and Mix of AI Units and Non-AI Units. Our work opens up new possibilities for building foundational software engineering tools based on LLMs, as opposed to traditional program analysis methods.

show abstract

Repair Is Nearly Generation: Multilingual Program Repair with LLMs

Cited by 8 publications

References 0 publications

Recommending Root-Cause and Mitigation Steps for Cloud Incidents using Large Language Models

Recommending Root-Cause and Mitigation Steps for Cloud Incidents using Large Language Models

Retrieving Multimodal Information for Augmented Generation: A Survey

Prompt-tuned Code Language Model as a Neural Knowledge Base for Type Inference in Statically-Typed Partial Code

Contact Info

Product

Resources

About