A language-agnostic model for semantic source code labeling

Abstract

Code search and comprehension have become more difficult in recent years due to the rapid expansion of available source code. Current tools lack a way to label arbitrary code at scale while maintaining up-to-date representations of new programming languages, libraries, and functionalities. Comprehensive labeling of source code enables users to search for documents of interest and obtain a high-level understanding of their contents. We use Stack Overflow code snippets and their tags to train a language-agnostic, deep convolutional neural network to automatically predict semantic labels for source code documents. On Stack Overflow code snippets, we demonstrate a mean area under ROC of 0.957 over a long-tailed list of 4,508 tags. We also manually validate the model outputs on a diverse set of unlabeled source code documents retrieved from Github, and obtain a top-1 accuracy of 86.6%. This strongly indicates that the model successfully transfers its knowledge from Stack Overflow snippets to arbitrary source code documents.

Multi-label Classification for Automatic Tag Prediction in the Context of Programming Challenges

“…e availability of more training data will then allow researchers to use more advanced text embeddings such as BERT [1] and XLNet [19] which are the current state-of-the-art in the NLP eld. Last but not least, di erent lines of work can also be tried such as avoiding preprocessing the dataset and learning character embeddings, similar to what was performed in [3].…”

A Literature Review of Using Machine Learning in Software Development Life Cycle Stages

“…Researchers in the articles have also suggested investigating further regarding the suitable metrics and loss functions employed in the evaluation of ML for SE-focused techniques, especially for multi-class classification problems [125].…”

A Visualization Tool for Analyzing the Suitability of Software Libraries via Their Code Repositories

“…To support requirements 2 and 3 (Changes in Code Content and Required Skills), the data pipeline uses Gelman et al's system [8] to generate a set of tags for each code file. These are semantic tags learned from Stack Overflow, such as c++, multithreading, or machine-learning.…”

“…By searching the open web we find that in September 2017 it was publicly stated that Theano is deprecated and new development will stop. 8 The timing of this announcement comes shortly before the large spike in commit activity near the end of 2017, which corresponds with the last major release of Theano. This announcement also comes around the time we see a sharp decline in the bus factor and the number of new issues.…”