Adapting Neural Text Classification for Improved Software Categorization

LeClair, Alexander; Eberhart, Zachary; McMillan, Collin

doi:10.1109/icsme.2018.00056

Cited by 30 publications

(21 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, as pointed out in Section III, an increasing number of research efforts are finding that code structure significantly improves code summarization. Evidence from Alon et al [37], LeClair et al [40], [64], and Haque et al [43] all point to how code structure seems to help code summarization tools broadly classify code behavior, even if good internal documentation is required to produce detailed summaries. We surmise that a way structure manifests itself in summaries is via the action word, and therefore ask RQ 3 .…”

Section: A Research Questionsmentioning

confidence: 99%

Action Word Prediction for Neural Source Code Summarization

Haque

Bansal

et al. 2021

2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)

Self Cite

View full text Add to dashboard Cite

Source code summarization is the task of creating short, natural language descriptions of source code. Code summarization is the backbone of much software documentation such as JavaDocs, in which very brief comments such as "adds the customer object" help programmers quickly understand a snippet of code. In recent years, automatic code summarization has become a high value target of research, with approaches based on neural networks making rapid progress. However, as we will show in this paper, the production of good summaries relies on the production of the action word in those summaries: the meaning of the example above would be completely changed if "removes" were substituted for "adds." In this paper, we advocate for a special emphasis on action word prediction as an important stepping stone problem towards better code summarizationcurrent techniques try to predict the action word along with the whole summary, and yet action word prediction on its own is quite difficult. We show the value of the problem for code summaries, explore the performance of current baselines, and provide recommendations for future research.

show abstract

Section: A Research Questionsmentioning

confidence: 99%

Action Word Prediction for Neural Source Code Summarization

Haque

Bansal

et al. 2021

2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Automatic documentation has a large potential impact on how so ware is developed and maintained. Not only would automatic documentation reduce the time and energy programmers spend reading and writing so ware, having a high level summary available has been shown to improve results in other SE tasks such as code categorization and code search [22,28].…”

Section: Problem Significance Scopementioning

confidence: 99%

Improved Code Summarization via a Graph Neural Network

LeClair¹,

Haque²,

Wu³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Automatic source code summarization is the task of generating natural language descriptions for source code. Automatic code summarization is a rapidly expanding research area, especially as the community has taken greater advantage of advances in neural network and AI technologies. In general, source code summarization techniques use the source code as input and outputs a natural language description. Yet a strong consensus is developing that using structural information as input leads to improved performance.e rst approaches to use structural information a ened the AST into a sequence. Recently, more complex approaches based on random AST paths or graph neural networks have improved on the models using a ened ASTs. However, the literature still does not describe the using a graph neural network together with source code sequence as separate inputs to a model. erefore, in this paper, we present an approach that uses a graph-based neural architecture that be er matches the default structure of the AST to generate these summaries. We evaluate our technique using a data set of 2.1 million Java method-comment pairs and show improvement over four baseline techniques, two from the so ware engineering literature, and two from machine learning literature.

show abstract

“…Code search [15,44,45,134,138], Code classification [30,74,154] code readability classification/prediction [105,117], Function type inferring [53,103] Code generation [35,115], Code Summarization [75,135], Code Decompilation [66,71] Code change generation [130], Data structure classification [108], Reverse execution [111] Design pattern detection [127], Technical debt detection [118], Story points prediction [21] Inconsistent method name refactoring [92], Stable patch detection [55] Defect Defect prediction [94,98,129,136,137,140], Vulnerability prediction [27,38,52,151] 8 24 Bug localization/detection [59,72,81,139,144,155] , Code repair [10,132,141] Code smell detection …”

Section: Codementioning

confidence: 99%

On the Replicability and Reproducibility of Deep Learning in Software Engineering

Liu,

Gao,

Xia

et al. 2020

Preprint

View full text Add to dashboard Cite

Deep learning (DL) techniques have gained significant popularity among software engineering (SE) researchers in recent years. This is because they can often solve many SE challenges without enormous manual feature engineering effort and complex domain knowledge. Although many DL studies have reported substantial advantages over other state-of-the-art models on effectiveness, they often ignore two factors: (1) replicability -whether the reported experimental result can be approximately reproduced in high probability with the same DL model and the same data; and (2) reproducibility -whether one reported experimental findings can be reproduced by new experiments with the same experimental protocol and DL model, but different sampled real-world data. Unlike traditional machine learning (ML) models, DL studies commonly overlook these two factors and declare them as minor threats or leave them for future work. This is mainly due to high model complexity with many manually set parameters and the time-consuming optimization process. In this study, we conducted a literature review on 93 DL studies recently published in twenty SE journals or conferences. Our statistics show the urgency of investigating these two factors in SE, where only 10.8% of the studies discussed any research questions affecting replicability and/or reproducibility. More than 74.2% of the studies do not even share source code and data to support the replicability of their complex models. Moreover, we re-ran four representative DL models in SE. Experimental results show the importance of replicability and reproducibility, where the reported performance of a DL model could not be replicated for an unstable optimization process. Reproducibility could be substantially compromised if the model training is not convergent, or if performance is sensitive to the size of vocabulary and testing data. It is therefore urgent for the SE community to provide a long-lasting link to a replication package, enhance DL-based solution stability and convergence, and avoid performance sensitivity on different sampled data.CCS Concepts: • Software and its engineering → Software maintenance tools.

show abstract

Adapting Neural Text Classification for Improved Software Categorization

Cited by 30 publications

References 52 publications

Action Word Prediction for Neural Source Code Summarization

Action Word Prediction for Neural Source Code Summarization

Improved Code Summarization via a Graph Neural Network

On the Replicability and Reproducibility of Deep Learning in Software Engineering

Contact Info

Product

Resources

About