Code Completion with Neural Attention and Pointer Networks

Li, Jian; Wang, Yue; Lyu, Michael R.; King, Irwin

doi:10.24963/ijcai.2018/578

Cited by 141 publications

(168 citation statements)

References 20 publications

Supporting

Mentioning

166

Contrasting

Unclassified

Order By: Relevance

“…The attention mechanism allows a neural language model to retrieve and make use of pertinent information in all previous hidden states, improving network retention. The mathematical details of the attention mechanism are described in previous work [38]. For attention, we use external memory M for previous hidden states, which is denoted as…”

Section: Proposed Lstm-attm Model Architecturementioning

confidence: 99%

Source Code Assessment and Classification Based on Estimated Error Probability Using Attentive LSTM Language Model and Its Application in Programming Education

2020

View full text Add to dashboard Cite

The rate of software development has increased dramatically. Conventional compilers cannot assess and detect all source code errors. Software may thus contain errors, negatively affecting end-users. It is also difficult to assess and detect source code logic errors using traditional compilers, resulting in software that contains errors. A method that utilizes artificial intelligence for assessing and detecting errors and classifying source code as correct (error-free) or incorrect is thus required. Here, we propose a sequential language model that uses an attention-mechanism-based long short-term memory (LSTM) neural network to assess and classify source code based on the estimated error probability. The attentive mechanism enhances the accuracy of the proposed language model for error assessment and classification. We trained the proposed model using correct source code and then evaluated its performance. The experimental results show that the proposed model has logic and syntax error detection accuracies of 92.2% and 94.8%, respectively, outperforming state-of-the-art models. We also applied the proposed model to the classification of source code with logic and syntax errors. The average precision, recall, and F-measure values for such classification are much better than those of benchmark models. To strengthen the proposed model, we combined the attention mechanism with LSTM to enhance the results of error assessment and detection as well as source code classification. Finally, our proposed model can be effective in programming education and software engineering by improving code writing, debugging, error-correction, and reasoning.

show abstract

Section: Proposed Lstm-attm Model Architecturementioning

confidence: 99%

Source Code Assessment and Classification Based on Estimated Error Probability Using Attentive LSTM Language Model and Its Application in Programming Education

2020

View full text Add to dashboard Cite

show abstract

“…[90][91][92][93][94][95][96][97][98][99][100][101][102] Paradigms of program synthesis. Paradigms [84,[103][104][105][106][107][108][109][110][111][112][113][114][115] Code completion and suggestion.…”

Section: [49]mentioning

confidence: 99%

Cognification of Program Synthesis—A Systematic Feature-Oriented Analysis and Future Direction

Subahi

2020

Computers

View full text Add to dashboard Cite

Program synthesis is defined as a software development step aims at achieving an automatic process of code generation that is satisfactory given high-level specifications. There are various program synthesis applications built on Machine Learning (ML) and Natural Language Processing (NLP) based approaches. Recently, there have been remarkable advancements in the Artificial Intelligent (AI) domain. The rise in advanced ML techniques has been remarkable. Deep Learning (DL), for instance, is considered an example of a currently attractive research field that has led to advances in the areas of ML and NLP. With this advancement, there is a need to gain greater benefits from these approaches to cognify synthesis processes for next-generation model-driven engineering (MDE) framework. In this work, a systematic domain analysis is conducted to explore the extent to the automatic generation of code can be enabled via the next generation of cognified MDE frameworks that support recent DL and NLP techniques. After identifying critical features that might be considered when distinguishing synthesis systems, it will be possible to introduce a conceptual design for the future involving program synthesis/MDE frameworks. By searching different research database sources, 182 articles related to program synthesis approaches and their applications were identified. After defining research questions, structuring the domain analysis, and applying inclusion and exclusion criteria on the classification scheme, 170 out of 182 articles were considered in a three-phase systematic analysis, guided by some research questions. The analysis is introduced as a key contribution. The results are documented using feature diagrams as a comprehensive feature model of program synthesis showing alternative techniques and architectures. The achieved outcomes serve as motivation for introducing a conceptual architectural design of the next generation of cognified MDE frameworks.

show abstract

“…Recent trends feature a boom by applying the Deep Learning Network (DNN) instead of LMs. However, while many studies [4], [13], [17], [22] have been developed to accommodate DNN in their code suggestion systems, a study [9] from Hellendoorn and Devanbu showed that carefully adapting n-gram models for source code can yield better performance than deep-learning models.…”

Section: Related Workmentioning

confidence: 99%

“…Existing solutions to code completion focus on using neural language models [4], [9], [13], [17], [22] or statistical language models [3], [16], [18], [19] learned from a large code base by modeling it into a natural language processing problem. However, they fail to utilize the user's input to narrow down the candidate list by proper relevance ranking.…”

Section: Introductionmentioning

confidence: 99%

Scope-aware Code Completion with Discriminative Modeling

Sheng

Xiao

Ishikawa

2019

Journal of Information Processing

View full text Add to dashboard Cite

Code completion is a traditional popular feature for API access in integrated development environments (IDEs). It not only frees programmers from remembering specific details about an API but also saves keystrokes and corrects typographical errors. Existing methods for code completion usually suggest APIs based on statistics in code bases described by language models. However, they neglect the fact that the user's input is also very useful for ranking, as the underlying patterns can be used to improve the accuracy of predictions of intended APIs. In this paper, we propose a novel method to improve the quality of code completion by incorporating the users' acronym-like input conventions and the APIs' scope context into a discriminative model. The users' input conventions are learned using a logistic regression model by extracting features from collected training data. The weights in the discriminative model are learned using a support vector machine (SVM). To improve the real-time efficiency of code completion, we employ a trie to index and store the scope context information. An efficient top-k algorithm is developed. Experiments show that our proposed method outperforms the baseline methods in terms of both effectiveness and efficiency.

show abstract

Code Completion with Neural Attention and Pointer Networks

Cited by 141 publications

References 20 publications

Source Code Assessment and Classification Based on Estimated Error Probability Using Attentive LSTM Language Model and Its Application in Programming Education

Source Code Assessment and Classification Based on Estimated Error Probability Using Attentive LSTM Language Model and Its Application in Programming Education

Cognification of Program Synthesis—A Systematic Feature-Oriented Analysis and Future Direction

Scope-aware Code Completion with Discriminative Modeling

Contact Info

Product

Resources

About