Foresight for ethical AI

Abstract

There is growing expectation that artificial intelligence (AI) developers foresee and mitigate harms that might result from their creations; however, this is exceptionally difficult given the prevalence of emergent behaviors that occur when integrating AI into complex sociotechnical systems. We argue that Naturalistic Decision Making (NDM) principles, models, and tools are well-suited to tackling this challenge. Already applied in high-consequence domains, NDM tools such as the premortem, and others, have been shown to uncover a reasonable set of risks of underlying factors that would lead to ethical harms. Such NDM tools have already been used to develop AI that is more trustworthy and resilient, and can help avoid unintended consequences of AI built with noble intentions. We present predictive policing algorithms as a use case, highlighting various factors that led to ethical harms and how NDM tools could help foresee and mitigate such harms.

RAD-XP: Tabletop Exercises for Eliciting Resilience Requirements for Sociotechnical Systems

“…After the TTX was complete and user stories had been generated, a team of three experts in resilience engineering rated the resultant user stories to determine whether they supported work system resilience or not (i.e., they were functional or non-functional requirements with no foreseeable impact to resilience). Each rater reviewed the set of user stories and provided a binary rating (1 = supports, 0 = does not support) for each of the resilience factors enumerated in the TRUSTS framework [8]. These ratings were not mutually exclusive, meaning that a single user story could be mapped to multiple resilience factors.…”

“…The adoption of agile development methods provides an opportunity for RAD tools such as RAD-XP that progressively reveal emergent requirements over the course of a technology's development. The ways a technology design can interact with work system operations are difficult to predict in advance [8]. As the prototype matures, the possible interactions become increasingly apparent and predictable.…”

“…We often focus on those goals and assume the technology will be adopted into use; however, any technology injected into the complex, time-honed dynamics of an established sociotechnical system is unlikely to be prepared for participating effectively in those dynamics. As such, it is unfortunately common that the introduction of new technologies to an existing sociotechnical system can have adverse unintended effects [6][7][8].…”

Digital ethicswashing: a systematic review and a process-perception-outcome framework

“…Further, in relation to emergent expectations toward AI developers to foresee and mitigate AI-related harms [26], more research is needed to address still unknown ethicswashing impacts that are difficult to assess and may only manifest along the AI system's lifecycle [47,111].…”