Participation is not a Design Fix for Machine Learning

Sloane, Mona; Moss, Emanuel; Awomolo, Olaitan; Forlano, Laura

doi:10.48550/arxiv.2007.02423

Cited by 29 publications

(36 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Future work should identify challenges for implementing QA in applied ML and AI development contexts, acknowledging the constraints that ML and AI development environments present. 1 Practitioners of these strategies must ensure that they engage with and compensate participants with care and respect, and that their input is meaningfully integrated into system design [38].…”

Section: Discussionmentioning

confidence: 99%

Qualitative Analysis for Human Centered AI

Papakyriakopoulos¹,

Watkins²,

Winecoff³

et al. 2021

Preprint

View full text Add to dashboard Cite

Human-centered artificial intelligence (AI) posits that machine learning and AI should be developed and applied in a socially aware way. In this article, we argue that qualitative analysis (QA) can be a valuable tool in this process, supplementing, informing, and extending the possibilities of AI models. We show this by describing how QA can be integrated in the current prediction paradigm of AI, assisting scientists in the process of selecting data, variables, and model architectures. Furthermore, we argue that QA can be a part of novel paradigms towards Human Centered AI. QA can support scientists and practitioners in practical problem solving and situated model development. It can also promote participatory design approaches, reveal understudied and emerging issues in AI systems, and assist policy making.

show abstract

Section: Discussionmentioning

confidence: 99%

Qualitative Analysis for Human Centered AI

Papakyriakopoulos¹,

Watkins²,

Winecoff³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…However, removing human agency from the training phase also removes any possibility for ethical accountability and oversight from the critical part of machine learning. In response we see a series of human-in-the-loop solutions that have tried to combine the engineering benefits of these approaches with the accountability of human agency (Gupta et al 2020, Sloane et al 2020). These include fairness analytics tools such as AIFairness360, the What-If tool, and explainability tools such as LIME.…”

Section: Machine Learning and Its Ethical Discontentmentioning

confidence: 99%

Can Machine Learning be Moral?

Sicart¹,

Shklovski²,

Jones³

2022

Preprint

View full text Add to dashboard Cite

“…Although having more diverse AI teams is critical, the "politics of inclusion" [22,92] of relying on marginalized practitioners may not be sufficient to effect systemic change given the dominant structural forces [e.g., 64] that might lead to those practitioners being ignored, tokenized, or fired [22,31,92]. Recent calls to foster greater engagement with affected communities when developing sociotechnical systems may serve as a counter to business imperatives [e.g., 20, 80], although it is important to be wary about extractive, tokenistic approaches [e.g., 3,4,74], which can unfairly burden members of marginalized groups [59].…”

Section: Implications Of Business Imperatives That Shape Disaggregate...mentioning

confidence: 99%

“…This question has a long historical resonance given Suchman's argument to attend to the specificities of place in technology development practices, including its micropolitics and cultural imaginaries, to avoid the reproduction of "neocolonial geographies of center and periphery" (and, more generally, to resist such neat binaries of center and periphery, local and global) [77]. More recently, Sloane et al argued that the ever-increasing drive to deploy AI systems at scale may be fundamentally at odds with calls to involve direct stakeholders in the development of those systems [74]. Our findings reveal implications of this tension, as many participants reported deploying AI systems in geographic contexts for which they have no processes for engaging with direct stakeholders.…”

Section: Implications Of Deploying Ai Systems At Scalementioning

confidence: 99%

Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support

Madaio¹,

Egede²,

Subramonyam³

et al. 2021

Preprint

View full text Add to dashboard Cite

Various tools and practices have been developed to support practitioners in identifying, assessing, and mitigating fairness-related harms caused by AI systems. However, prior research has highlighted gaps between the intended design of these tools and practices and their use within particular contexts, including gaps caused by the role that organizational factors play in shaping fairness work. In this paper, we investigate these gaps for one such practice: disaggregated evaluations of AI systems, intended to uncover performance disparities between demographic groups. By conducting semi-structured interviews and structured workshops with thirty-three AI practitioners from ten teams at three technology companies, we identify practitioners' processes, challenges, and needs for support when designing disaggregated evaluations. We find that practitioners face challenges when choosing performance metrics, identifying the most relevant direct stakeholders and demographic groups on which to focus, and collecting datasets with which to conduct disaggregated evaluations. More generally, we identify impacts on fairness work stemming from a lack of engagement with direct stakeholders, business imperatives that prioritize customers over marginalized groups, and the drive to deploy AI systems at scale.

show abstract

Participation is not a Design Fix for Machine Learning

Cited by 29 publications

References 0 publications

Qualitative Analysis for Human Centered AI

Qualitative Analysis for Human Centered AI

Can Machine Learning be Moral?

Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support

Contact Info

Product

Resources

About