An empirical study of bots in software development: characteristics and challenges from a practitioner’s perspective

Abstract

Software engineering bots ś automated tools that handle tedious tasks ś are increasingly used by industrial and open source projects to improve developer productivity. Current research in this area is held back by a lack of consensus of what software engineering bots (DevBots) actually are, what characteristics distinguish them from other tools, and what benefits and challenges are associated with DevBot usage. In this paper we report on a mixed-method empirical study of DevBot usage in industrial practice. We report on findings from interviewing 21 and surveying a total of 111 developers. We identify three different personas among DevBot users (focusing on autonomy, chat interfaces, and łsmartnessž), each with different definitions of what a DevBot is, why developers use them, and what they struggle with. We conclude that future DevBot research should situate their work within our framework, to clearly identify what type of bot the work targets, and what advantages practitioners can expect. Further, we find that there currently is a lack of generalpurpose łsmartž bots that go beyond simple automation tools or chat interfaces. This is problematic, as we have seen that such bots, if available, can have a transformative effect on the projects that use them. CCS CONCEPTS • Software and its engineering → Software notations and tools.

Identifying bot activity in GitHub pull request and issue comments

“…Two different individuals could disagree on the interpretation of comments as being produced by bots. As a consequence, a more fine-grained classification of (types of) bots would be needed, building further on the work by Erlenhov et al [20] who identified three types of bot personas based on their autonomy, chat interface, and smartness. It would be definitely interesting to explore how the binary classifier we propose can be generalised to detect these different types of bot personas.…”

Bots Don't Mind Waiting, Do They? Comparing the Interaction With Automatically and Manually Created Pull Requests

“…The main driver for the adoption of bots, according to a study by Erlenhov et al [9], is to increase productivity, although this aspect is seen differently by different DevBot users. We already knew from a study by Meyer et al [10] that developers perceive their workday as productive if they could complete tasks without significant interruptions.…”

“…We already knew from a study by Meyer et al [10] that developers perceive their workday as productive if they could complete tasks without significant interruptions. Interestingly, all DevBot users in Erlenhov's study had to some extent issues with interruptions or noise produced by bots: "A good bot waits until a developer is ready for feedback" [9].…”

A Longitudinal Analysis of Bloated Java Dependencies

“…Software Bots. Erlenhov et al [9] perform an empirical study about the interaction between practitioners and software bots. They found that there is currently a lack of general-purpose smart bots that go beyond simple automation tools, such as dependency version updating.…”