Choosing your weapons: On sentiment analysis tools for software engineering research

Jongeling, Robbert; Datta, Subhajit; Serebrenik, Alexander

doi:10.1109/icsm.2015.7332508

Cited by 82 publications

(44 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This paper builds on our previous work (Jongeling et al 2015). The current submission extends it by reporting on replication of two recent studies (Section 5).…”

Section: Replications and Negative Resultsmentioning

confidence: 65%

“…This paper builds on our previous work (Jongeling et al 2015). The current submission extends it by reporting on a follow-up study (Section 3.3), replication of two recent studies (Section 5) as well presenting a more elaborate discussion of the related work below.…”

Section: Related Workmentioning

confidence: 76%

“…In our previous work (Jongeling et al 2015) and in the current submission we perform a series of quantitative analyses aiming at evaluation whether the choice of the sentiment analysis tool can affect the validity of the software engineering results. A complementary approach to evaluating the applicability of sentiment analysis tools to software engineering data has been followed by Novielli et al (2015) that performed a qualitative analysis of STACK OVERFLOW posts and compared the results of SENTISTRENGTH with those obtained by manual evaluation.…”

Section: Sentiment Analysis Toolsmentioning

confidence: 99%

See 2 more Smart Citations

On negative results when using sentiment analysis tools for software engineering research

Sarkar²,

et al. 2017

Self Cite

View full text Add to dashboard Cite

Recent years have seen an increasing attention to social aspects of software engineering, including studies of emotions and sentiments experienced and expressed by the software developers. Most of these studies reuse existing sentiment analysis tools such as SENTISTRENGTH and NLTK. However, these tools have been trained on product reviews and movie reviews and, therefore, their results might not be applicable in the software engineering domain. In this paper we study whether the sentiment analysis tools agree with the sentiment recognized by human evaluators (as reported in an earlier study) as well as with each other. Furthermore, we evaluate the impact of the choice of a sentiment analysis tool on software engineering studies by conducting a simple study of differences in issue resolution times for positive, negative and neutral texts. We repeat the study for seven datasets (issue trackers and STACK OVERFLOW questions) and different sentiment analysis tools and observe that the disagreement between the tools can lead to diverging conclusions. Finally, we perform two replications of previously published studies and observe that the results of those studies cannot be confirmed when a different sentiment analysis tool is used.

show abstract

“…This paper builds on our previous work (Jongeling et al 2015). The current submission extends it by reporting on replication of two recent studies (Section 5).…”

Section: Replications and Negative Resultsmentioning

confidence: 65%

Section: Related Workmentioning

confidence: 76%

Section: Sentiment Analysis Toolsmentioning

confidence: 99%

See 1 more Smart Citation

On negative results when using sentiment analysis tools for software engineering research

Sarkar²,

et al. 2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…Therefore, their performances may underachieve the expectations or in the worst case lead to wrong results. Limited to sentiment analysis, Jongeling et al [6] showed that general purpose tools such as SentiStrength and NLTK were unreliable for assessing sentiments in technical prose within issue comments.…”

Section: Introductionmentioning

confidence: 99%

The emotional side of software developers in JIRA

Ortu

Murgia

Destefanis

et al. 2016

Proceedings of the 13th International Conference on Mining Software Repositories

101

View full text Add to dashboard Cite

Issue tracking systems store valuable data for testing hypotheses concerning maintenance, building statistical prediction models and (recently) investigating developer a↵ec-tiveness. For the latter, issue tracking systems can be mined to explore developers emotions, sentiments and politeness -a↵ects for short. However, research on a↵ect detection in software artefacts is still in its early stage due to the lack of manually validated data and tools.In this paper, we contribute to the research of a↵ects on software artefacts by providing a labeling of emotions present on issue comments. We manually labeled 2,000 issue comments and 4,000 sentences written by developers with emotions such as love, joy, surprise, anger, sadness and fear. Labeled comments and sentences are linked to software artefacts reported in our previously published dataset (containing more than 1K projects, more than 700K issue reports and more than 2 million issue comments). The enriched dataset presented in this paper allows the investigation of the role of a↵ects in software development.

show abstract

“…It is obvious that this is a fitting estimate of the sentiment in the review. As inferred by Jongeling et al in [12], the scores obtained from SentiStrength do not necessarily tally with those from other sentiment analysis APIs. We reassure the same empirically for drug review data, by drawing up a comparison of normalized sentiment scores Table II) for review sample sizes ≥ 30.…”

Section: Extraction and Feasibility Of Usingmentioning

confidence: 77%

PeopleSave: Recommending effective drugs through web crowdsourcing

Majethia

Mishra

Singhal

et al. 2016

2016 8th International Conference on Communication Systems and Networks (COMSNETS)

View full text Add to dashboard Cite

In this paper, we describe PeopleSave -a drug recommendation and feedback system for doctors on the basis of contextual patient reviews crowd-sourced from the Internet. Unlike other systems proposed in the past, we filter information sources to check for crowdsourcing feasibility and then assess the drug's effectiveness based on its reported detrimental effect on a patient. This helps in eliminating certain drugs that would almost certainly have an adverse effect on the patient's health and thereby obtain a set of recommendable drugs. These recommendations are further refined by analyzing the sentiment behind the opinions of patients who have been administered these drugs in the past. The resultant set of prescribable drugs agrees with those suggested by the consulted physicians for the considered sample set of diabetes patients. The critical assessment of the prototype system for Diabetes Type II drugs by both doctors and patients also reiterates the need for a feedback system that can possibly go a long way in improving patient experience of a drug. This leads us to conclude that PeopleSave, as a combination of the recommendation system prototype and the proposed feedback system, can be successful in improving the process of prescription of medicines for a varied range of medical conditions.

show abstract

Choosing your weapons: On sentiment analysis tools for software engineering research

Cited by 82 publications

References 26 publications

On negative results when using sentiment analysis tools for software engineering research

On negative results when using sentiment analysis tools for software engineering research

The emotional side of software developers in JIRA

PeopleSave: Recommending effective drugs through web crowdsourcing

Contact Info

Product

Resources

About