Development and Application of Sentiment Analysis Tools in Software Engineering: A Systematic Literature Review

Abstract

Software development is a collaborative task and, hence, involves different persons. Research has shown the relevance of social aspects in the development team for a successful and satisfying project closure. Especially the mood of a team has been proven to be of particular importance. Thus, project managers or project leaders want to be aware of situations in which negative mood is present to allow for interventions. So-called sentiment analysis tools offer a way to determine the mood based on text-based communication. In this paper, we present the results of a systematic literature review of sentiment analysis tools developed for or applied in the context of software engineering. Our results summarize insights from 80 papers with respect to (1) the application domain, (2) the purpose, (3) the used data sets, (4) the approaches for developing sentiment analysis tools and ( 5) the difficulties researchers face when applying sentiment analysis in the context of software projects. According to our results, sentiment analysis is frequently applied to open-source software projects, and most tools are based on support-vector machines. Despite the frequent use of sentiment analysis in software engineering, there are open issues, e.g., regarding the identification of irony or sarcasm, pointing to future research directions. CCS CONCEPTS• Software and its engineering → Collaboration in software development; • Human-centered computing → Collaborative and social computing systems and tools.

An empirical study of IoT security aspects at sentence-level in developer textual discussions

“…Our selection of these five transformer models is based on previous findings in SE classification tasks. BERT has shown massive improvement over other deep models for sentiment analysis tools in SE [53] [19]. Other variants of BERT, i.e., RoBERTa, XLNet, etc., are also used in the SE classification task.…”

On the Subjectivity of Emotions in Software Projects: How Reliable are Pre-Labeled Data Sets for Sentiment Analysis?

“…According to a recent literature review by Obaidi and Klünder [7], sentiment analysis has been in the focus of several researchers. For example, Novielli et al [19] describe how they developed a gold standard data set to support the ongoing research on sentiment analysis in software engineering.…”

“…Obaidi and Klünder [7] examined the development and application of sentiment analysis in software engineering through a systematic literature review (n = 80 papers). Among other results, their findings show that both GitHub and Stack Overflow are among the top three data sources used for sentiment analysis.…”

“…This, in turn, has a negative influence on the development team as a whole and the software project. As a result, the research field of sentiment analysis (sometimes also referred to as emotion mining) in software engineering has grown steadily over the past few years [7]. Nowadays, there exist multiple tools using different techniques to apply sentiment analysis to data sources from software engineering, including JIRA [8], GitHub [9], and Stack Overflow [10].…”

“…Nowadays, there exist multiple tools using different techniques to apply sentiment analysis to data sources from software engineering, including JIRA [8], GitHub [9], and Stack Overflow [10]. A recent systematic literature study [7] shows that almost all of these tools use some form of machine learning, requiring some kind of training data to allow the algorithm to learn which statements, comments, and texts evoke which sentiment polarity, and to be able to measure the performance of these tools.…”