Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI

Abstract

Many organizations have published principles intended to guide the ethical development and deployment of AI systems; however, their abstract nature makes them difficult to operationalize. Some organizations have therefore produced AI ethics checklists, as well as checklists for more specific concepts, such as fairness, as applied to AI systems. But unless checklists are grounded in practitioners' needs, they may be misused. To understand the role of checklists in AI ethics, we conducted an iterative co-design process with 48 practitioners, focusing on fairness. We co-designed an AI fairness checklist and identified desiderata and concerns for AI fairness checklists in general. We found that AI fairness checklists could provide organizational infrastructure for formalizing ad-hoc processes and empowering individual advocates. We discuss aspects of organizational culture that may impact the efficacy of such checklists, and highlight future research directions.

What does it mean to embed ethics in data science? An integrative approach based on microethics and virtues

“…Data ethics modules do not have yet a precise identity, but a list of courses discussing ethical issues related to AI and data science 3 shows that many courses are shaped along the lines of the characteristics of bioethics/RCR that we outlined above. Teaching courses in this way reflects a macroethical approach that simply imports in pedagogy the same issues of applicability outlined above (McNamara et al 2018;Madaio et al 2020;Vakkuri et al 2020).…”

Z-Inspection®: A Process to Assess Trustworthy AI

“…[45], [46], [75]. In Madaio et al [76], they identify that the beneficial outcome of implementing an AI ethics checklist may be to prompt discussion and reflection that might otherwise not take place. Checklists themselves may not be sufficient to influence practitioners' decisions, rather, checklists need to address the stakeholder roles and organizational procedures, thus empowering practitioners while following certain processes.…”

Community-in-the-loop: towards pluralistic value creation in AI, or—why AI needs business ethics

“…In particular questions of explainable AI [34][35][36][37][38][39] or issues of fairness [40][41][42][43][44][45][46][47] have emerged as productive fields of research. But also more governance-oriented approaches to the practical implementation of ethical AI play an important part, for example with regard to a professional code of conduct for developers [48,49], a more direct involvement of ethicists in the development of AI systems [50,51], or in terms of checklists [52,53], adapted internal structures [54], suitable impact assessment frameworks [55] and auditing processes [56] or a value-based AI label [57]. Finally, perspectives from the law concern the ethical design of AI at the interface with regulatory issues [58][59][60][61][62].…”

“…To the extent that there is a tendency to implement ethics in the sense of "better building" by means of technical solutions, mainly developers and data scientists are assumed responsible for ethical action. In addition to the application of appropriate technical measures, this is reflected, for example, in the development of professional ethics [48,49], the teaching of ethics to AI practitioners [87] or tools such as checklists [52,53,88] directed at developers and data scientists. But also critical contributions, which rather belong to an emerging third wave of AI ethics, sometimes tend to argue with a focus on individuals as relevant actors for ethical AI [10].…”