TreeGen: A Tree-Based Transformer Architecture for Code Generation

Sun, Zeyu; Zhu, Qihao; Xiong, Yingfei; Sun, Yican; Mou, Lili; Zhang, Lu

doi:10.1609/aaai.v34i05.6430

Cited by 130 publications

(76 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Typically, propose TRANX, which introduces ASTs as intermediate representations of codes and has become the most influential Seq2Tree model. Then, Sun et al (2019Sun et al ( , 2020 respectively explore CNN and Transformer 2020) exploit external knowledge to enhance neural code generation model. Generally, all these Seq2Tree models generate ASTs in pre-order traversal, which, how-ever, is not suitable to handle all multi-branch AST nodes.…”

Section: Related Workmentioning

confidence: 99%

“…* Equal contribution † Corresponding author models, and obtains great success (Ling et al, 2016;Lapata, 2016, 2018;Rabinovich et al, 2017;Yin and Neubig, 2017Hayati et al, 2018;Sun et al, 2019Sun et al, , 2020Wei et al, 2019;Shin et al, 2019;Xu et al, 2020;Xie et al, 2021). Specifically, an encoder is first used to learn word-level semantic representations of the input NL description.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Exploring Dynamic Selection of Branch Expansion Orders for Code Generation

Jiang¹,

Zhou²,

Meng³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Due to the great potential in facilitating software development, code generation has attracted increasing attention recently. Generally, dominant models are Seq2Tree models, which convert the input natural language description into a sequence of tree-construction actions corresponding to the pre-order traversal of an Abstract Syntax Tree (AST). However, such a traversal order may not be suitable for handling all multi-branch nodes. In this paper, we propose to equip the Seq2Tree model with a context-based Branch Selector, which is able to dynamically determine optimal expansion orders of branches for multibranch nodes. Particularly, since the selection of expansion orders is a non-differentiable multi-step operation, we optimize the selector through reinforcement learning, and formulate the reward function as the difference of model losses obtained through different expansion orders. Experimental results and in-depth analysis on several commonly-used datasets demonstrate the effectiveness and generality of our approach. We have released our code at https:

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Exploring Dynamic Selection of Branch Expansion Orders for Code Generation

Jiang¹,

Zhou²,

Meng³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

show abstract

“…The authors evaluate their model in machine translation and text classification. Sun et al (2020) develop a tree-structured transformer encoder-decoder architecture for code generation. Here, the tree structure is based on the code syntax.…”

Section: Related Workmentioning

confidence: 99%

Recursive Tree-Structured Self-Attention for Answer Sentence Selection

Mrini¹,

Farcas²,

Nakashole³

2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Syntactic structure is an important component of natural language text. Recent topperforming models in Answer Sentence Selection (AS2) use self-attention and transfer learning, but not syntactic structure. Tree structures have shown strong performance in tasks with sentence pair input like semantic relatedness. We investigate whether tree structures can boost performance in AS2. We introduce the Tree Aggregation Transformer: a novel recursive, tree-structured self-attention model for AS2. The recursive nature of our model is able to represent all levels of syntactic parse trees with only one additional self-attention layer. Without transfer learning, we establish a new state of the art on the popular TrecQA and WikiQA benchmark datasets. Additionally, we evaluate our method on four Community Question Answering datasets, and find that tree-structured representations have limitations with noisy user-generated text. We conduct probing experiments to evaluate how our models leverage tree structures across datasets. Our findings show that the ability of treestructured models to successfully absorb syntactic information is strongly correlated with a higher performance in AS2.

show abstract

“…Hu et al [19] proposed LSTM based code completion tool by inducing tokens at character and token levels, thereby reducing vocabulary size. Sun et al [51] developed neural code generation model that implements attention mechanism and tree-based AST reader. These prior efforts targeted imperative languages.…”

Section: Related Workmentioning

confidence: 99%

On the naturalness of hardware descriptions

Lee

Nie

et al. 2020

Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Softw

View full text Add to dashboard Cite

Mining software repositories (MSR) has been shown effective for extracting data used to improve various software engineering tasks, including code completion, code repair, code search, and code summarization. Despite a large body of work on MSR, researchers have focused almost exclusively on repositories that contain code written in imperative programming languages, such as Java and C/C++. Unlike prior work, in this paper, we focus on mining publicly available hardware descriptions (HDs) written in hardware description languages (HDLs), such as VHDL. HDLs have unique syntax and semantics compared to popular imperative languages, and learningbased tools available to hardware designers are well behind those used in other application domains. We assembled large HD corpora consisting of source code written in several HDLs and report on their characteristics. Our language model evaluation reveals that HDs possess a high level of naturalness similar to software written in imperative languages. Further, by utilizing our corpora, we built several deep learning models for automated code completion in VHDL; our models take into account unique characteristics of HDLs, including similarities of nearby concurrent signal assignment statements, in-built concurrency, and the frequently used signal types. These characteristics led to more effective neural models, achieving a BLEU score of 37.3, an 8-14-point improvement over rule-based and neural baselines. CCS CONCEPTS • Hardware → Hardware description languages and compilation; • Software and its engineering → Software maintenance tools.

show abstract

TreeGen: A Tree-Based Transformer Architecture for Code Generation

Cited by 130 publications

References 14 publications

Exploring Dynamic Selection of Branch Expansion Orders for Code Generation

Exploring Dynamic Selection of Branch Expansion Orders for Code Generation

Recursive Tree-Structured Self-Attention for Answer Sentence Selection

On the naturalness of hardware descriptions

Contact Info

Product

Resources

About