GypSum: Learning Hybrid Representations for Code Summarization

Abstract

Code summarization with deep learning has been widely studied in recent years. Current deep learning models for code summarization generally follow the principle in neural machine translation and adopt the encoder-decoder framework, where the encoder learns the semantic representations from source code and the decoder transforms the learnt representations into human-readable text that describes the functionality of code snippets. Despite they achieve the new state-of-the-art performance, we notice that current models often either generate less fluent summaries, or fail to capture the core functionality, since they usually focus on a single type of code representations. As such we propose GypSum, a new deep learning model that learns hybrid representations using graph attention neural networks and a pre-trained programming and natural language model. We introduce particular edges related to the control flow of a code snippet into the abstract syntax tree for graph construction, and design two encoders to learn from the graph and the token sequence of source code, respectively. We modify the encoder-decoder sublayer in the Transformer's decoder to fuse the representations and propose a dual-copy mechanism to facilitate summary generation. Experimental results demonstrate the superior performance of GypSum over existing code summarization models. CCS CONCEPTS• Computing methodologies → Neural networks; • Software and its engineering → Documentation.

PyScribe–Learning to describe python code

“…62 Although the objective of CodeBERT does not include generation tasks, it can be modified by introducing a Transformer decoder for code comment generation. 64 Table 9 presents the results of PyScribe and the baselines. We refer to the baseline performances reported by Choi et al 34 The overall results show that the recent Transformer-based approaches NeuralCodeSum 15 and mAST+GCN 34 outperform the previous works based on RNNs.…”

Structure and Sequence Aligned Code Summarization with Prefix and Suffix Balanced Strategy

“…Alon et al [ 15 ] extracted multiple tree paths from root node to terminal node based on ASTs as the code representations as structural information for summary generation. Currently, Graph Neural Networks (GNNs) achieve great performance on graph data (e.g., protein prediction, knowledge graph), so researchers [ 17 , 18 ] consider constructing AST into a graph by adding additional edge relations and utilizing GNNs to capture structural information achieving great results.…”