“That’s important, but...”: How Computer Science Researchers Anticipate Unintended Consequences of Their Research Innovations

Abstract

Computer science research has led to many breakthrough innovations but has also been scrutinized for enabling technology that has negative, unintended consequences for society. Given the increasing discussions of ethics in the news and among researchers, we interviewed 20 researchers in various CS sub-disciplines to identify whether and how they consider potential unintended consequences of their research innovations. We show that considering unintended consequences is generally seen as important but rarely practiced. Principal barriers are a lack of formal process and strategy as well as the academic practice that prioritizes fast progress and publications. Drawing on these findings, we discuss approaches to support researchers in routinely considering unintended consequences, from bringing diverse perspectives through community participation to increasing incentives to investigate potential consequences. We intend for our work to pave the way for routine explorations of the societal implications of technological innovations before, during, and after the research process. CCS CONCEPTS• Human-centered computing → HCI theory, concepts and models.

Farsight: Fostering Responsible AI Awareness During AI Application Prototyping

“…Prior work analyzed these broader impacts statements, finding convergence around a set of topics such as risks to privacy and bias, but often lacking concrete specifics or strategies for mitigation [8,99,127,167]. However, prior work suggests that many CS researchers may not have the training, resources, or inclination to engage in this type of anticipatory work [45,175], suggesting that new tools, training, and processes, are needed to support researchers and developers in engaging in anticipatory work in ways that are integrated into their research practices. More recently, researchers have proposed a framework that uses LLMs to anticipate harms for classifiers by generating stakeholders and vignettes for a given scenario [24], evaluating this framework through interviews with responsible AI researchers.…”

“…Guide users in imagining use cases. Existing research highlights the challenges faced by ML practitioners when attempting to anticipate the uses of their ML-powered applications and how different individuals or groups may be affected [20,45,103,171]. Confirming this, software engineer U6 noted "You don't really know how your tool could be used, so it's really hard to envision what harms would be."…”

“…Many users of AI prompt-based prototyping interfaces [e.g., 70,133,184,204], or "AI prototypers" [cf. 84] do not have experience in AI or computer science, which can lead to challenges in anticipating the consequences of their AI applications [143]-a difficult task even for computer science faculty and AI researchers [20,45]. Furthermore, LLMs demonstrate a wide range of capabilities that are continually being discovered across Farsight provides a range of in situ widgets for these tools, helping AI prototypers envision the potential harms of their AI applications during an early prototyping stage.…”

Data Ethics Emergency Drill: A Toolbox for Discussing Responsible AI for Industry Teams

“…They ask for tools that fit into their resource constraints, as ML practices outside of big tech may not have the bandwidth to carry out some of the responsible AI investigations of larger companies [29]. Finally, technologists require structures for discussing and reflecting on the ethical implications of their work [10,16,24,28]. Conflicts between values themselves and between different professionals will arise naturally, so to address this organisations will need to "(...) mobilize resources to create safe spaces and encourage explicit disagreements among practitioners positively [and] enable them to constantly question RAI values (...)" [59, p.13].…”