Code Structure Guided Transformer for Source Code Summarization

Shuzheng, Gao,; Gao, Cuiyun; He, Yan; Zeng, Jichuan; Nie, Lun Yiu; Xia, Xinhui; Lyu, Michael R.

doi:10.48550/arxiv.2104.09340

Cited by 7 publications

(9 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The last is embedding ASTs using GNNs [33,37], and here we take GCN as an example. As the number of GCN layers increases, each node can aggregate a more extensive range of information from its neighbors, thus focusing on a broader range of local features.…”

Section: Motivating Examplementioning

confidence: 99%

“…They used the AST sequences obtained via the SBT method and the embedding representation obtained using GCN to extract the structure of ASTs. • SG-Trans: Gao et al [37] injected local semantic information and global syntactic structure into the self-attentive module in the Transformer as an inductive bias to better capture the hierarchical features of source code.…”

Section: Baselinesmentioning

confidence: 99%

“…With the emergence of the Transformer model [29], Ahmad et al [30] improved it to generate summaries for both Java and Python language. Due to the power of the Transformer model, much work has subsequently emerged around it [37,49,56,57].…”

Section: Related Work 61 Source Code Summarizationmentioning

confidence: 99%

See 2 more Smart Citations

M2TS: Multi-Scale Multi-Modal Approach Based on Transformer for Source Code Summarization

Gao¹,

Lyu²

2022

Preprint

View full text Add to dashboard Cite

Source code summarization aims to generate natural language descriptions of code snippets. Many existing studies learn the syntactic and semantic knowledge of code snippets from their token sequences and Abstract Syntax Trees (ASTs). They use the learned code representations as input to code summarization models, which can accordingly generate summaries describing source code. Traditional models traverse ASTs as sequences or split ASTs into paths as input. However, the former loses the structural properties of ASTs, and the latter destroys the overall structure of ASTs. Therefore, comprehensively capturing the structural features of ASTs in learning code representations for source code summarization remains a challenging problem to be solved. In this paper, we propose M2TS, a Multi-scale Multi-modal approach based on Transformer for source code Summarization. M2TS uses a multi-scale AST feature extraction method, which can extract the structures of ASTs more completely and accurately at multiple local and global levels. To complement missing semantic information in ASTs, we also obtain code token features, and further combine them with the extracted AST features using a cross modality fusion method that not only fuses the syntactic and contextual semantic information of source code, but also highlights the key features of each modality. We conduct experiments on two Java and one Python datasets, and the experimental results demonstrate that M2TS outperforms current state-of-the-art methods. We release our code at https://github.com/TranSMS/M2TS.

show abstract

Section: Motivating Examplementioning

confidence: 99%

Section: Baselinesmentioning

confidence: 99%

See 1 more Smart Citation

M2TS: Multi-Scale Multi-Modal Approach Based on Transformer for Source Code Summarization

Gao¹,

Lyu²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…They replace the original positional encoding in the Transformer with the relative positional encoding, which encodes the pairwise relationship between the tokens in the token sequence. As an improvement to Transformer, Gao et al [18] propose to exploit code structural property and introduce constraints to the multi-head self-attention module. Zhang et al [47] and Wei et al [43] both propose a retrieval-based method, which improves summary generation by leveraging the comments of similar code snippets.…”

Section: Related Workmentioning

confidence: 99%

GypSum: Learning Hybrid Representations for Code Summarization

Wang,

Dong,

et al. 2022

Preprint

View full text Add to dashboard Cite

Code summarization with deep learning has been widely studied in recent years. Current deep learning models for code summarization generally follow the principle in neural machine translation and adopt the encoder-decoder framework, where the encoder learns the semantic representations from source code and the decoder transforms the learnt representations into human-readable text that describes the functionality of code snippets. Despite they achieve the new state-of-the-art performance, we notice that current models often either generate less fluent summaries, or fail to capture the core functionality, since they usually focus on a single type of code representations. As such we propose GypSum, a new deep learning model that learns hybrid representations using graph attention neural networks and a pre-trained programming and natural language model. We introduce particular edges related to the control flow of a code snippet into the abstract syntax tree for graph construction, and design two encoders to learn from the graph and the token sequence of source code, respectively. We modify the encoder-decoder sublayer in the Transformer's decoder to fuse the representations and propose a dual-copy mechanism to facilitate summary generation. Experimental results demonstrate the superior performance of GypSum over existing code summarization models. CCS CONCEPTS• Computing methodologies → Neural networks; • Software and its engineering → Documentation.

show abstract

“…Comment synthesis is now an active research area, including many projects such as CodeNN [30], DeepCom [26], Astattgru [40], C BERT [18], Rencos [74], SecNN [42], PLBART [1], CoTexT [54], ProphetNet-X [55], NCS [2], Code2seq [7], Re 2 Com [71], and many more [19,24,25,27,28,38,39,41,49,50,66,67,69,70,72,73]. All these approaches rely on datasets of aligned code-comment pairs.…”

Section: Introductionmentioning

confidence: 99%

Multilingual training for Software Engineering

Ahmed,

Devanbu

2021

Preprint

View full text Add to dashboard Cite

Well-trained machine-learning models, which leverage large amounts of open-source software data, have now become an interesting approach to automating many software engineering tasks. Several SE tasks have all been subject to this approach, with performance gradually improving over the past several years with better models and training methods. More, and more diverse, clean, labeled data is better for training; but constructing good-quality datasets is time-consuming and challenging. Ways of augmenting the volume and diversity of clean, labeled data generally have wide applicability. For some languages (e.g., Ruby) labeled data is less abundant; in others (e.g., JavaScript) the available data maybe more focused on some application domains, and thus less diverse. As a way around such data bottlenecks, we present evidence suggesting that human-written code in different languages (which performs the same function), is rather similar, and particularly preserving of identifier naming patterns; we further present evidence suggesting that identifiers are a very important element of training data for software engineering tasks. We leverage this rather fortuitous phenomenon to find evidence that available multilingual training data (across different languages) can be used to amplify performance. We study this for 3 different tasks: code summarization, code retrieval, and function naming. We note that this dataaugmenting approach is broadly compatible with different tasks, languages, and machine-learning models.

show abstract

Code Structure Guided Transformer for Source Code Summarization

Cited by 7 publications

References 0 publications

M2TS: Multi-Scale Multi-Modal Approach Based on Transformer for Source Code Summarization

M2TS: Multi-Scale Multi-Modal Approach Based on Transformer for Source Code Summarization

GypSum: Learning Hybrid Representations for Code Summarization

Multilingual training for Software Engineering

Contact Info

Product

Resources

About