From Preference Elicitation to Participatory ML: A Critical Survey &amp; Guidelines for Future Research

Feffer, Michael; Skirpan, Michael; Lipton, Zachary; Heidari, Hoda

doi:10.1145/3600211.3604661

Cited by 4 publications

(4 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the ICML workshop, several authors have reviewed Participatory ML projects and demonstrated a gap between the field's goals and actual practices. Feffer et al [33] and Robertson and Salehi [64] argue that many Participatory ML projects are limited to developing computational methods to elicit participant preferences, rather than re-imagining participants as co-designers throughout the ML process. Similarly, Corbett et al [24] and Delgado et al [25] draw on Arnstein's Ladder of Citizen Participation [3] to note the lack of consistency in what gets classified as a participatory approach in ML development, and a tendency towards informing or consulting with participants, rather than engaging in equitable partnerships.…”

Section: Power and Participatory MLmentioning

confidence: 99%

“…Some authors have proposed frameworks or guiding questions for Participatory ML, to shift practice towards more equitable partnerships with participants [24,26,33], and to ensure participation 1 Data subjects are defined in this paper as people whose personal data can be used to identify them. Data subjects include end users, defined in this paper as the direct users of an ML artefact or system.…”

Section: Power and Participatory MLmentioning

confidence: 99%

“…Participatory approaches have been enacted across each stage of ML design and development-from problem formulation to model evaluation-and include collaborative approaches to construct datasets [62,73], design and validate ML algorithms [57,72], and guide advocacy for algorithmic accountability [47,63]. At the same time, several authors have raised concerns about "participation-washing" [70], cooptation of participatory work [7], and the limited evidence across Participatory ML projects of equitable partnerships with participants [24,26,33].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

From Fitting Participation to Forging Relationships: The Art of Participatory ML

Cooper,

Zafiroglu

2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Participatory machine learning (ML) encourages the inclusion of end users and people affected by ML systems in design and development processes. We interviewed 18 participation brokers-individuals who facilitate such inclusion and transform the products of participants' labour into inputs for an ML artefact or system-across a range of organisational settings and project locations. Our findings demonstrate the inherent challenges of integrating messy contextual information generated through participation with the structured data formats required by ML workflows and the uneven power dynamics in project contexts. We advocate for evolution in the role of brokers to more equitably balance value generated in Participatory ML projects for design and development teams with value created for participants. To move beyond 'fitting' participation to existing processes and empower participants to envision alternative futures through ML, brokers must become educators and advocates for end users, while attending to frustration and dissent from indirect stakeholders.

show abstract

Section: Power and Participatory MLmentioning

confidence: 99%

Section: Power and Participatory MLmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

From Fitting Participation to Forging Relationships: The Art of Participatory ML

Cooper,

Zafiroglu

2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

show abstract

“…This is the case of the second category of model monitoring, namely unsupervised monitoring. Estimating the performance and fairness of AI models in the absence of labelled data is a very challenging task with impossibility theorems delimiting the work (Garg et al, 2022;Zhang et al, 2021;Fang et al, 2022).…”

Section: Monitoring Biasmentioning

confidence: 99%

Policy advice and best practices on bias and fairness in AI

Alvarez,

Colmenarejo,

Elobaid

et al. 2024

Ethics Inf Technol

View full text Add to dashboard Cite

The literature addressing bias and fairness in AI models (fair-AI) is growing at a fast pace, making it difficult for novel researchers and practitioners to have a bird’s-eye view picture of the field. In particular, many policy initiatives, standards, and best practices in fair-AI have been proposed for setting principles, procedures, and knowledge bases to guide and operationalize the management of bias and fairness. The first objective of this paper is to concisely survey the state-of-the-art of fair-AI methods and resources, and the main policies on bias in AI, with the aim of providing such a bird’s-eye guidance for both researchers and practitioners. The second objective of the paper is to contribute to the policy advice and best practices state-of-the-art by leveraging from the results of the NoBIAS research project. We present and discuss a few relevant topics organized around the NoBIAS architecture, which is made up of a Legal Layer, focusing on the European Union context, and a Bias Management Layer, focusing on understanding, mitigating, and accounting for bias.

show abstract

Sketching AI Concepts with Capabilities and Examples: AI Innovation in the Intensive Care Unit

Yildirim,

Zlotnikov,

Sayar

et al. 2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

From Preference Elicitation to Participatory ML: A Critical Survey & Guidelines for Future Research

Cited by 4 publications

References 40 publications

From Fitting Participation to Forging Relationships: The Art of Participatory ML

From Fitting Participation to Forging Relationships: The Art of Participatory ML

Policy advice and best practices on bias and fairness in AI

Sketching AI Concepts with Capabilities and Examples: AI Innovation in the Intensive Care Unit

Contact Info

Product

Resources

About