SeTransformer: A Transformer-Based Code Semantic Parser for Code Comment Generation

Li, Zheng; Wu, Yonghao; Peng, Bin; Chen, Xiang; Sun, Zeyu; Liu, Yong; Becevel, Anna

doi:10.1109/tr.2022.3154773

Cited by 20 publications

(24 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deepcom [6] utilizes the traditional SBT and Hybrid-DeepCom [11] created comments on Java methods through an encoder-decoder structure with attention. Yang et al [7] used Sim_SBT, and SeTransformer [4] used improved structure-based traversal (ISBT) to deduplicate SBT and improve its representation. They used the transformer [22] instead of an RNN, which has a long-term dependency on long input data.…”

Section: Learning-based Code Summarizationmentioning

confidence: 99%

“…Haiduc et al [18] used only lexical information using code tokens for comment generation. Hybrid-DeepCom [11], ComFormer [7], SeCNN [24], and SeTransformer [4] use the source code structure information as well as the source code in comment generation. These methods use lexical and syntactic information together.…”

Section: A Source Code Information 1) Code Sequencementioning

confidence: 99%

“…During software development and maintenance, developers usually take a significant amount of time to understand programs. Commented codes are easier to understand than uncommented codes, and high-quality code comments can effectively improve program comprehension [1]- [4]. However, writing comments and keeping them up to date requires substantial time and effort [5].…”

Section: Introductionmentioning

confidence: 99%

“…State-of-the-art models [4], [7] with recurrent neural network (RNN) [8], Seq2Seq [9], and transformer [10] to gener-ate comments generally used structural information obtained by various abstract syntax tree (AST) traversal methods of source code, such as the structure-based traversal (SBT) method. This indicates that simultaneously using the lexical and syntactic information of a programming language increases the accuracy of translation from the source code to the comment.…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, we emphasize the importance of aligning lexical and syntactic information. Many studies [4], [6], [7], [11] have used SBT, where the order and number of tokens differ from those of the corresponding keywords in the original source code. In NLP, it has been commonly observed that machine translation performs better between languages with similar word order.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

ALSI-Transformer: Transformer-Based Code Comment Generation With Aligned Lexical and Syntactic Information

Park

Kim

2023

IEEE Access

View full text Add to dashboard Cite

Code comments explain the operational process of a computer program and increase the long-term productivity of programming tasks such as debugging and maintenance. Therefore, developing methods that automatically generate natural language comments from programming code is required. With the development of deep learning, various excellent models in the natural language processing domain have been applied for comment generation tasks, and recent studies have improved performance by simultaneously using the lexical information of the code token and the syntactical information obtained from the syntax tree. In this paper, to improve the accuracy of automatic comment generation, we introduce a novel syntactic sequence, Code-Aligned Type sequence (CAT), to align the order and length of lexical and syntactic information, and we propose a new neural network model, Aligned Lexical and Syntactic information-Transformer (ALSI-Transformer), based on a transformer that encodes the aligned multimodal information with convolution and embedding aggregation layers. Through in-depth experiments, we compared ALSI-Transformer with current baseline methods using standard machine translation metrics and demonstrate that the proposed method achieves state-of-the-art performance in code comment generation.

show abstract

Section: Learning-based Code Summarizationmentioning

confidence: 99%

Section: A Source Code Information 1) Code Sequencementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

ALSI-Transformer: Transformer-Based Code Comment Generation With Aligned Lexical and Syntactic Information

Park

Kim

2023

IEEE Access

View full text Add to dashboard Cite

show abstract

Semantic similarity loss for neural source code summarization

Su,

McMillan

2024

J Software Evolu Process

View full text Add to dashboard Cite

This paper presents a procedure for and evaluation of using a semantic similarity metric as a loss function for neural source code summarization. Code summarization is the task of writing natural language descriptions of source code. Neural code summarization refers to automated techniques for generating these descriptions using neural networks. Almost all current approaches involve neural networks as either standalone models or as part of a pretrained large language models, for example, GPT, Codex, and LLaMA. Yet almost all also use a categorical cross‐entropy (CCE) loss function for network optimization. Two problems with CCE are that (1) it computes loss over each word prediction one‐at‐a‐time, rather than evaluating a whole sentence, and (2) it requires a perfect prediction, leaving no room for partial credit for synonyms. In this paper, we extend our previous work on semantic similarity metrics to show a procedure for using semantic similarity as a loss function to alleviate this problem, and we evaluate this procedure in several settings in both metrics‐driven and human studies. In essence, we propose to use a semantic similarity metric to calculate loss over the whole output sentence prediction per training batch, rather than just loss for each word. We also propose to combine our loss with CCE for each word, which streamlines the training process compared to baselines. We evaluate our approach over several baselines and report improvement in the vast majority of conditions.

show abstract

Distilled GPT for source code summarization

Su,

McMillan

2024

Autom Softw Eng

View full text Add to dashboard Cite

SeTransformer: A Transformer-Based Code Semantic Parser for Code Comment Generation

Cited by 20 publications

References 43 publications

ALSI-Transformer: Transformer-Based Code Comment Generation With Aligned Lexical and Syntactic Information

ALSI-Transformer: Transformer-Based Code Comment Generation With Aligned Lexical and Syntactic Information

Semantic similarity loss for neural source code summarization

Distilled GPT for source code summarization

Contact Info

Product

Resources

About