Deep Code-Comment Understanding and Assessment

Abstract

Code comments are a key software component for program comprehension and software maintainability. High-quality code and comments are urgently needed by data-driven models widely used in tasks like code summarization. Many existing approaches for assessing the quality of comments are machine learning based classification algorithms or rely on heuristic rules. These approaches are difficult to capture the complicated features of text data and are often limited in accuracy, efficiency, and generalization ability. In this paper, we convert the quality assessment of code comments into a classification problem based on the multi-input neural network. We summarize the input, the code and comments, into vectors using the attention-based Bi-LSTM model and the weighted GloVe model, respectively, and concatenate the code vectors and the comment vectors as the input of the Multiple-Layer Perceptron classifier for the comment quality assessment. Experimental results show that our approach, in general, outperforms the previous technique, on both our labeled dataset and the public dataset, with the F1-score of 96.91% and 91.90%, respectively. Using the training set and the testing set from distinct sources, our approach can still achieve reasonable performance, which demonstrates its generalization ability.INDEX TERMS Code comment, source code, multi-input neural network, text classification.

Leveraging machine learning for software redocumentation—A comprehensive comparison of methods in practice

“…In comparison to our work, the works presented so far in the field of comment classification uses only classical techniques of ML, for example, decision trees. To our knowledge, the only other DL‐based approach in the field is presented by Wang et al 10 . To assess coherence of a method's implementation with a method's lead comment Wang et al present a framework that vectorizes code and comment tokens using the GloVe model.…”

“…To our knowledge, the only other DL-based approach in the field is presented by Wang et al. 10 To assess coherence of a method's implementation with a method's lead comment Wang et al present a framework that vectorizes code and comment tokens using the GloVe model. To summarize token vectors, vectors for code are summarized by using a trained Bi-LSTM model to focus on important information in source code.…”

“…To our knowledge the only work which applies DL to comment classification is reported by Wang et al. 10 Wang et al as well treat the quality assessment of comments as a classification problem and apply DL using a GloVe model for comment vectorization and a Bi-LSTM model to summarize token vectors of source code. Their approach achieves the f1 score of 96.91% and 91.90%, respectively, on two different datasets.…”

“…Current practices for classification of source code comments in deep learning (DL) are rare. The only example we found is on evaluating the coherence or noncoherence of comments with source code statements 10 …”

“…The only example we found is on evaluating the coherence or noncoherence of comments with source code statements. 10 Within this article, we investigate different ML algorithms, also covering recent DL concepts, and compare them with each other as well as with originally developed heuristics and state-of-the-art approaches using evaluation metrics on real-world examples. The research work described in this article is based on our previous work, 11 where we already presented the concrete industrial setting and our developed heuristics for solving the binary classification problem, whether a source code comment is to be integrated into the documentation of a legacy software system or not.…”