Fast and Memory-Efficient Neural Code Completion

Svyatkovskiy, A.; Lee, Sebastian; Hadjitofi, Anna; Riechert, Maik; Franco, Juliana Rocha; Allamanis, Miltiadis

doi:10.48550/arxiv.2004.13651

Cited by 7 publications

(11 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similar DNNs can assist programmers 1609,1610 . For example, by predictive source code completion [1611][1612][1613][1614][1615][1616] , and by generating source code to map inputs to target outputs 1617 or from labels describing desired source code 1618 . Text generating DNNs can also help write scientific papers.…”

Section: Discussionmentioning

confidence: 99%

Review: Deep Learning in Electron Microscopy

Ede

2020

Preprint

View full text Add to dashboard Cite

Deep learning is transforming most areas of science and technology, including electron microscopy. This review paper offers a practical perspective aimed at developers with limited familiarity. For context, we review popular applications of deep learning in electron microscopy. Following, we discuss hardware and software needed to get started with deep learning and interface with electron microscopes. We then review neural network components, popular architectures, and their optimization. Finally, we discuss future directions of deep learning in electron microscopy.

show abstract

Section: Discussionmentioning

confidence: 99%

Review: Deep Learning in Electron Microscopy

Ede

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Machine Learning for Programming Languages. In recent years, deep learning methods have been used to tackle various tasks in software engineering, with a particular interest in bug detection and repair (Wang et al, 2017;Allamanis et al, 2018;Tarlow et al, 2019;Murali et al, 2020;Dinella et al, 2020;Yasunaga & Liang, 2020; and code completion (Li et al, 2018;Kim et al, 2020;Svyatkovskoy et al, 2020). Unsupervised pre-training methods for code based on BERT (Kanade et al, 2020;, BART (Ahmad et al, 2021) or other objectives tailored to source code Roziere et al, 2021) have shown strong results on benchmarks such as CodeXGLUE (Lu et al, 2021).…”

Section: Related Workmentioning

confidence: 99%

Leveraging Automated Unit Tests for Unsupervised Code Translation

Rozière¹,

Zhang²,

Charton³

et al. 2021

Preprint

View full text Add to dashboard Cite

With little to no parallel data available for programming languages, unsupervised methods are well-suited to source code translation. However, the majority of unsupervised machine translation approaches rely on back-translation, a method developed in the context of natural language translation and one that inherently involves training on noisy inputs. Unfortunately, source code is highly sensitive to small changes; a single token can result in compilation failures or erroneous programs, unlike natural languages where small inaccuracies may not change the meaning of a sentence. To address this issue, we propose to leverage an automated unit-testing system to filter out invalid translations, thereby creating a fully tested parallel corpus. We found that fine-tuning an unsupervised model with this filtered data set significantly reduces the noise in the translations so-generated, comfortably outperforming the state-of-the-art for all language pairs studied. In particular, for Java → Python and Python → C++ we outperform the best previous methods by more than 16% and 24% respectively, reducing the error rate by more than 35%.

show abstract

“…b) Lack of fusion ranking methods to combine results from multi code completion algorithms. Due to the different interests of research, such as the different semantic representation of programming language [21]- [23] and the different completion targets (API calls [12], [24]- [27], AST node [5], [28], entire line [20]), each code completion algorithm has its distinct characteristics. In addition, the traditional code completion also has a different type of results sorted in alphabetical order.…”

Section: Imentioning

confidence: 99%

“…If we perform code segmentation by token granularity, the long tail effect of the programming language will be more evident than the general natural language. Therefore, we adopt a finer-grained sub-token segmentation method [27] to further segment tokens when encountering underscore or camel case naming rules. In addition, we filter out the sub-tokens that only appear in one project, and the length is smaller than 5.…”

Section: A Frequency Strategiesmentioning

confidence: 99%

Toward Less Hidden Cost of Code Completion with Acceptance and Ranking Models

Li¹,

Huang²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

Code completion is widely used by software developers to provide coding suggestions given a partially written code snippet. Apart from the traditional code completion methods, which only support single token completion at minimal positions, recent studies show the ability to provide longer code completion at more flexible positions. However, such frequently triggered and longer completion results reduce the overall precision as they generate more invalid results. Moreover, different studies are mostly incompatible with each other. Thus, it is vital to develop an ensemble framework that can combine results from multiple models to draw merits and offset defects of each model. This paper conducts a coding simulation to collect data from code context and different code completion models and then apply the data in two tasks. First, we introduce an acceptance model which can dynamically control whether to display completion results to the developer. It uses simulation features to predict whether correct results exist in the output of these models. Our best model reduces the percentage of false-positive completion from 55.09% to 17.44%. Second, we design a fusion ranking scheme that can automatically identify the priority of the completion results and reorder the candidates from multiple code completion models. This scheme is flexible in dealing with various models, regardless of the type or the length of their completion results. We integrate this ranking scheme with two frequency models and a GPT-2 styled language model, along with the acceptance model to yield 27.80% and 37.64% increase in TOP1 and TOP5 accuracy, respectively. In addition, we propose a new code completion evaluation metric, Benefit-Cost Ratio(BCR), taking into account the benefit of keystrokes saving and hidden cost of completion list browsing, which is closer to real coder experience scenario.

show abstract

Fast and Memory-Efficient Neural Code Completion

Cited by 7 publications

References 0 publications

Review: Deep Learning in Electron Microscopy

Review: Deep Learning in Electron Microscopy

Leveraging Automated Unit Tests for Unsupervised Code Translation

Toward Less Hidden Cost of Code Completion with Acceptance and Ranking Models

Contact Info

Product

Resources

About