Sentiment Analysis of Free/Open Source Developers: Preliminary Findings From a Case Study

Abstract

<jats:p>O desenvolvimento de software é uma atividade intensive em esforço humano. Assim, a forma como os desenvolvedores encaram suas tarefas é de suam importância. Em um ambiente como o usual em projetos de FOSS (free/open source software) em que profissionais (desenvolvedores pagos) compartilham os esforços de desenvolvimento com voluntários, a moral da comunidade de desenvolvedores e usuários é fundamental. Neste artigo, apresentamos uma análise preliminary utilizando técnicas de análise de sentimentos realizada em um projeto de FOSS. Para isso, executamos a mineração da lista de endereços eletrônicos de um projeto e aplicamos as técnicas propostas aos participantes mais relevantes. Embora a aplicação seja limitada, no momento atual, experamos que essa experiência possa ser benéfica no future para determiner situações que possam afetar os desenvolvedores ou o projeto, tais como baixa produtividade, abandono do projeto ou bifurcação do projeto, entre outras.</jats:p>

On negative results when using sentiment analysis tools for software engineering research

“…NLTK has been applied in earlier software engineering studies (Pletea et al 2014;Rousinopoulos et al 2014). NLTK uses a simple bag of words model and returns for each text three probabilities: a probability of the text being negative, one of it being neutral and one of it being positive.…”

“…Sentiment polarity analysis has been recently applied in the software engineering context to study commit comments in GitHub (Guzman et al 2014), GitHub discussions related to security (Pletea et al 2014), productivity in Jira issue resolution , activity of contributors in Gentoo (Garcia et al 2013), classification of user reviews for maintenance and evolution (Panichella et al 2015) and evolution of developers' sentiments in the openSUSE Factory (Rousinopoulos et al 2014). It has also been suggested when assessing technical candidates on the social web (Capiluppi et al 2013).…”

“…It has also been suggested when assessing technical candidates on the social web (Capiluppi et al 2013). Not surprisingly, all the aforementioned software engineering studies with the notable exception of the work by Panichella et al (2015), reuse the existing sentiment polarity tools, e.g., (Pletea et al 2014) and (Rousinopoulos et al 2014) use NLTK, while (Garcia et al 2013;Guzman and Bruegge 2013;Guzman et al 2014;Novielli et al 2015) and opted for SENTISTRENGTH. While the reuse of the existing tools facilitated the application of the sentiment polarity analysis techniques in the software engineering domain, it also introduced a commonly recognized threat to validity of the results obtained: those tools have been trained on non-software engineering related texts such as movie reviews or product reviews and might misidentify (or fail to identify) polarity of a sentiment in a software engineering artefact such as a commit comment (Guzman et al 2014;Pletea et al 2014).…”

Choosing your weapons: On sentiment analysis tools for software engineering research

“…Sentiment polarity analysis has been recently applied in the software engineering context to study commit comments in GitHub [9], GitHub discussions related to security [10], productivity in Jira issue resolution [11], activity of contributors in Gentoo [12] and evolution of developers' sentiments in the openSUSE Factory [13]. It has also been suggested when assessing technical candidates on the social web [14].…”

“…It has also been suggested when assessing technical candidates on the social web [14]. Not surprisingly, all the aforementioned software engineering studies reuse the existing sentiment polarity tools, e.g., Guzman et al [15], [9] and Rousinopoulos et al [13] use NLTK, while Garcia et al [12], Pletea et al [10] and Ortu et al [11] opted for SentiStrength. While the reuse of the existing tools facilitated the application of the sentiment polarity analysis techniques in the software engineering domain, it also introduced a commonly recognized threat to validity of the results obtained: those tools have been trained on nonsoftware engineering related texts such as movie reviews or product reviews and might misidentify (or fail to identify) polarity of a sentiment in a software engineering artefact such as a commit comment [9], [10].…”