A Neural Network Based Intelligent Support Model for Program Code Completion

Abstract

In recent years, millions of source codes are generated in different languages on a daily basis all over the world. A deep neural network-based intelligent support model for source code completion would be a great advantage in software engineering and programming education fields. Vast numbers of syntax, logical, and other critical errors that cannot be detected by normal compilers continue to exist in source codes, and the development of an intelligent evaluation methodology that does not rely on manual compilation has become essential. Even experienced programmers often find it necessary to analyze an entire program in order to find a single error and are thus being forced to waste valuable time debugging their source codes. With this point in mind, we proposed an intelligent model that is based on long short-term memory (LSTM) and combined it with an attention mechanism for source code completion. Thus, the proposed model can detect source code errors with locations and then predict the correct words. In addition, the proposed model can classify the source codes as to whether they are erroneous or not. We trained our proposed model using the source code and then evaluated the performance. All of the data used in our experiments were extracted from Aizu Online Judge (AOJ) system. The experimental results obtained show that the accuracy in terms of error detection and prediction of our proposed model approximately is 62% and source code classification accuracy is approximately 96% which outperformed a standard LSTM and other state-of-the-art models. Moreover, in comparison to state-of-the-art models, our proposed model achieved an interesting level of success in terms of error detection, prediction, and classification when applied to long source code sequences. Overall, these experimental results indicate the usefulness of our proposed model in software engineering and programming education arena.

A Bidirectional LSTM Language Model for Code Evaluation and Repair

“…Experimental results, obtained by tuning the various hyper parameters and settings of the network, showed that both models achieved better results for bug detection and code completion in comparison with other related models. In [20,23], the authors proposed error detection, logic error detection, and the classification of source codes on the basis of an LSTM model. Both approaches used an attention mechanism that enhanced model scalability.…”

“…A variety of source code and software engineering methods have been proposed, such as source code classification [12,13], code clone detection [14,15], defect prediction [16], program repair [17,18], and code completion [19,20]. Recently, natural language processing (NLP) has been used in a number of domains, including speech recognition, language processing, and machine translation.…”

“…However, while these models may be useful for relatively short, simple codes, they are considerably less effective for long, complex codes. Today, deep neural network models are being used for language modeling due to their ability to consider long input sequences, and deep neural network-based language models are being developed for source code bug detection, logic error detection, and code completion [20,[22][23][24][25]. Recurrent neural networks (RNNs) have been used but are less effective due to gradient vanishing or exploding [26].…”

A Survey on Machine Learning Techniques for Source Code Analysis

“…The rest of the approaches employed existing benchmarks and datasets. Rahman et al [255] trained their proposed model using the data extracted from Aizu Online Judge (aoj) system. Liu et al [191], Liu et al [192] performed experiments on three real-world datasets to evaluate the effectiveness of their model when compared with the state-of-the-art approaches.…”

Impact of Practical Skills on Academic Performance: A Data-Driven Analysis

“…Many universities have created their own automated program assessment (APA) systems for programming courses to accelerate students' learning [12]- [14]. As a result, a large number of programmingrelated submission logs are created every day by OJ or APA systems in various organizations worldwide, which can be valuable resources for research and analysis [15], [16]. Therefore, this research aims to use programming-related resources (submission logs) for empirical research and analysis.…”