Pythia: AI-assisted Code Completion System

Svyatkovskiy, A.; Zhao, Ying; Fu, Sheng-Yu; Sundaresan, Neel

doi:10.1145/3292500.3330699

Cited by 115 publications

(82 citation statements)

References 11 publications

(12 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, in addition to the local context in LHS and previous concurrent assignments, we utilize the types of signals as global context as they are extracted from the port and local signal declarations at the beginning of entity/architecture definitions. This idea aligns with the recent work that uses types for code completion in imperative languages [27,53].…”

Section: Neural Architecturesupporting

confidence: 69%

“…Raychev et al [45] proposed an RNN language model for completing holes in partial programs with the most likely sequences of API method calls. Svyatkovskiy et al [53] proposed a neural code completion model that incorporates type information, instantiated as a tool called Pythia for Python. Hellendoorn et al [16] applied several code completion tools on real-world data and analyzed the real-world efficacy of those tools.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

On the naturalness of hardware descriptions

Lee

Nie

et al. 2020

Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Softw

View full text Add to dashboard Cite

Mining software repositories (MSR) has been shown effective for extracting data used to improve various software engineering tasks, including code completion, code repair, code search, and code summarization. Despite a large body of work on MSR, researchers have focused almost exclusively on repositories that contain code written in imperative programming languages, such as Java and C/C++. Unlike prior work, in this paper, we focus on mining publicly available hardware descriptions (HDs) written in hardware description languages (HDLs), such as VHDL. HDLs have unique syntax and semantics compared to popular imperative languages, and learningbased tools available to hardware designers are well behind those used in other application domains. We assembled large HD corpora consisting of source code written in several HDLs and report on their characteristics. Our language model evaluation reveals that HDs possess a high level of naturalness similar to software written in imperative languages. Further, by utilizing our corpora, we built several deep learning models for automated code completion in VHDL; our models take into account unique characteristics of HDLs, including similarities of nearby concurrent signal assignment statements, in-built concurrency, and the frequently used signal types. These characteristics led to more effective neural models, achieving a BLEU score of 37.3, an 8-14-point improvement over rule-based and neural baselines. CCS CONCEPTS • Hardware → Hardware description languages and compilation; • Software and its engineering → Software maintenance tools.

show abstract

Section: Neural Architecturesupporting

confidence: 69%

Section: Related Workmentioning

confidence: 99%

On the naturalness of hardware descriptions

Lee

Nie

et al. 2020

Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Softw

View full text Add to dashboard Cite

show abstract

“…Since these languages by definition have a grammar, syntax, and known relationships between entities, they offer enticing opportunities for an even deeper probing of NLP models and tasks. Beyond theoretical importance, many NLP tasks have practical utility in software development environments: language modeling or generation can be used for code completion (Raychev et al, 2014;Bruch et al, 2009;Svyatkovskiy et al, 2019Svyatkovskiy et al, , 2020, translation/summarization to generate documentation or natural language summaries (Moreno et al, 2013;Scalabrino et al, 2017;Wan et al, 2018;Alon et al, 2018) or even summarize a set of code changes (Moreno et al, 2014), translation and grammar error correction to patch and detect bugs (Zhai et al, 2019), and joint embedding of code and natural language for code search (Husain et al, 2019;Gu et al, 2018).…”

Section: Introductionmentioning

confidence: 99%

PyMT5: multi-mode translation of natural language and Python code with transformers

Clement¹,

Drain²,

Timcheck³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

Self Cite

View full text Add to dashboard Cite

Simultaneously modeling source code and natural language has many exciting applications in automated software development and understanding. Pursuant to achieving such technology, we introduce PYMT5, the PYTHON method text-to-text transfer transformer, which is trained to translate between all pairs of PYTHON method feature combinations: a single model that can both predict whole methods from natural language documentation strings (docstrings) and summarize code into docstrings of any common style. We present an analysis and modeling effort of a large-scale parallel corpus of 26 million PYTHON methods and 7.7 million method-docstring pairs, demonstrating that for docstring and method generation, PYMT5 outperforms similarlysized auto-regressive language models (GPT2) which were English pre-trained or randomly initialized. On the CODE-SEARCHNET test set, our best model predicts 92.1% syntactically correct method bodies, achieved a BLEU score of 8.59 for method generation and 16.3 for docstring * Corresponding author † Work done during a Microsoft internship generation (summarization), and achieved a ROUGE-L F-score of 24.8 for method generation and 36.7 for docstring generation.

show abstract

“…For next token prediction, a popular method is to feed the source sequence tokens into an RNN (or LSTM) [8 ]- [10], [22]. An RNN embeds the input tokens into a vector: x t = emb(wt), where wt is the source token seen at the t 'th time step.…”

Section: B Seqrnnmentioning

confidence: 99%

Code Prediction by Feeding Trees to Transformers

Kim

Zhao

Tian

et al. 2021

2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

106

103

View full text Add to dashboard Cite

Code prediction, more specifically autocomplete, has become an essential feature in modern IDEs. Autocomplete is more effective when the desired next token is at (or close to) the top of the list of potential completions offered by the IDE at cursor position. This is where the strength of the underlying machine learning system that produces a ranked order of potential completions comes into play.We advance the state-of-the-art in the accuracy of code prediction (next token prediction) used in autocomplete systems. Our work uses Transformers as the base neural architecture. We show that by making the Transformer architecture aware of the syntactic structure of code, we increase the margin by which a Transformer-based system outperforms previous systems. With this, it outperforms the accuracy of several state-of-the-art next token prediction systems by margins ranging from 14% to 18%.We present in the paper several ways of communicating the code structure to the Transformer, which is fundamentally built for processing sequence data. We provide a comprehensive experimental evaluation of our proposal, along with alternative design choices, on a standard Python dataset, as well as on a company internal Python corpus. Our code and data preparation pipeline will be available in open source.

show abstract

Pythia: AI-assisted Code Completion System

Cited by 115 publications

References 11 publications

On the naturalness of hardware descriptions

On the naturalness of hardware descriptions

PyMT5: multi-mode translation of natural language and Python code with transformers

Code Prediction by Feeding Trees to Transformers

Contact Info

Product

Resources

About