Fine-Tuning GPT-2 to Patch Programs, Is It Worth It?

Abstract

The application of Articial Intelligence (AI) in the Software Engineering (SE) eld is always a bit delayed compared to state-ofthe-art research results. While the Generative Pre-trained Transformer (GPT-2) model was published in 2018, only a few recent works used it to SE tasks. One of such task is Automated Program Repair (APR), where the applied technique should nd a x to software bugs without human intervention. One problem emerges here: the creation of proper training data is resource intensive and requires several hours of additional work from researchers. The sole reason of it is that training a model to repair programs automatically requires both the buggy program and the xed one in large scale and presumably in an already pre-processed form. There are currently few such databases, so teaching and ne-tuning models is not an easy task. In this work we wanted to investigate how the GPT-2 model performs when it is not ne-tuned for the APR task, compered to when it is ne-tuned. From previous work we already know that the GPT-2 model can automatically generate patches for buggy programs, although the literature lacks of studies where no ne-tuning has taken place. For the sake of experiment we evaluated the GPT-2 model out-of-the-box and also ne-tuned it before the evaluation on 1559 JavaSript code snippets. Based on out results we can conclude that although the ne-tuned model was able to learn how to write syntactically correct source code almost on every attempt, the non-ne-tuned model lacked some of these positive features.

MARGEN: Marathi Question Answering Generative Conversation Model

“…3 learnt an internal representation of the English dialect that can at that point be utilized to extricate highlights valuable for downstream errands. The demonstrate is best at what it was pretrained for be that as it may, which is producing writings from a provoke [20,21,22].…”

“…The coming about dataset (called WebText) weights 40GB of writings but has not been freely discharged. Engineer can discover a list of the beat 1,000 spaces display on Hugging Face [22].…”