Tool Choice Matters: JavaScript Quality Assurance Tools and Usage Outcomes in GitHub Projects

Heard it through the Gitvine: an empirical study of tool diffusion across the npm ecosystem

“…The only notable exception is Codeclimate, where the effect is reversed, i.e., if a repository has adopted a competing tool, then it is highly likely to also adopt Codeclimate. One explanation could be that Codeclimate has more features than Codecov and Coveralls [41]. In addition, badges for Codecov and Coveralls look the same, while the badge for Codeclimate is visually distinct.…”

“…Moreover, while we investigated whether quality assurance tools spread differently than non-tool badges, in reality even tools within the same task class may vary in terms of configuration overhead and features [41]; from the perspective of toolsmiths, it would worth studying which usability features allow more effective diffusion.…”

“…Software engineering researchers and practitioners are often frustrated by the low adoption of practices and tools, such as static analysis, refactoring tools, program comprehension, or security testing, even though many of them are generally perceived to be beneficial. Researchers have found a wide range of technical and social pain points in tool adoption, such as false positives in static analysis tools [e.g., 39,64,71], missing trust in correctness [e.g., 74], crypting tool messages [e.g., 38,39], slow response times [e.g., 80], lack of workflow integration [e.g., 36,41,64,86], lack of collaboration support [39], lack of management buy-in [e.g., 18,80], overwhelming configuration effort [e.g., 25,36,39,71,80], and simply a lack of knowledge about tools [e.g., 60,74,87]. In response, most software engineering research has focused on technical solutions, such as improving functionality, accuracy, and performance [e.g., 8,64,72], improving usability [e.g., 38,46,51,72,79], and improving discoverability through recommendation mechanisms or process integration [e.g., 49,64,87].…”

“…For example, one could choose between Travis, Circle, and Appveyor for their continuous integration needs on GitHub, just to name a few. At the same time, developers rarely choose more than one competing tool for a given task, and rarely switch tools once they have made a choice [41,80]. Institutionalists also note that getting an additional signal for the same source is seen as redundant and has diminishing returns [44].…”

“…On "social coding" platforms like GitHub, Gitlab, and Bit-Bucket observability is high. Such platforms offer a high level of transparency [16,17], a wealth of channels through which developers and software projects can become interconnected [45], and a diversity of signals [41,73] through which people can get exposed to each other's practices and tools. For example, for GitHub open-source projects using continuous integration, the pass or fail status of each build is automatically displayed alongside each commit in the user interface as cross ( ) or check signs ( ), which link to a specific tool's website (e.g., Travis, Circle) and the build logs produced by that tool; project maintainers can also choose to display additional repository badges (e.g., , ) indicating the continuous integration build outcome in real time, also linking to the specific tool and corresponding build log.…”