Automatically Generating Release Notes with Content Classification Models

Abstract

Release notes are admitted as an essential technical document in software maintenance. They summarize the main changes, e.g. bug fixes and new features, that have happened in the software since the previous release. Manually producing release notes is a time-consuming and challenging task. For that reason, sometimes developers neglect to write release notes. For example, we collect data from GitHub with over 1900 releases, and among them, 37% of the release notes are empty. To mitigate this problem, we propose an automatic release notes generation approach by applying the text summarization techniques, i.e. TextRank. To improve the keyword extraction method of traditional TextRank, we integrate the GloVe word embedding technique with TextRank. After generating release notes automatically, we apply machine learning algorithms to classify the release note contents (or sentences). We classify the contents into six categories, e.g. bug fixes and performance improvements, to represent the release notes better for users. We use the evaluation metric, e.g. ROUGE, to evaluate the automatically generated release notes. We also compare the performance of our technique with two popular extractive algorithms, e.g. Luhn’s and latent semantic analysis (LSA). Our evaluation results show that the improved TextRank method outperforms the two algorithms.

Practitioners' expectations on automated release note generation techniques

“…It is important to note that in some instances, the milestones tab and associated labels might not be updated or fully completed. Addressing this challenge could involve the implementation of text classification models aimed at effectively categorizing the contents of release notes 14 …”

“…Addressing this challenge could involve the implementation of text classification models aimed at effectively categorizing the contents of release notes. 14 The process of classification holds several advantages. It enables the prioritization of diverse information types, positioning critical updates in a prominent manner, and presenting less pivotal details in a structured format.…”

Automatic Analysis of Available Source Code of Top Artificial Intelligence Conference Papers

“…Hassan et al [61] introduce a usability requirements extraction approach based on a usability keywords repository created by the usability category of ISO 9126 and ISO 25010. Nath & Roy [62] propose a release notes generation method by employing text summarization technologies. They then classify the generated notes into 6 categories for friendly reading.…”