Many hands make many fingers to point: challenges in creating accountable AI

Slota, Stephen C.; Fleischmann, Kenneth R.; Greenberg, Sherri R.; Verma, Nitin; Cummings, Brenna; Li, Lan; Shenefiel, Chris

doi:10.1007/s00146-021-01302-0

Cited by 13 publications

(6 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Systemic and implicit biases such as racism and other forms of discrimination can inadvertently manifest in AI through the data used in training, as well as through the institutional policies and practices underlying how AI is commissioned, developed, deployed, and used. Statistical/algorithmic and human cognitive and perceptual biases enter the engineering and modeling processes themselves, and an inability to properly validate model performance leaves these biases exposed during deployment [62,103,112,113]. These biases collide with the cognitive biases of the individuals interacting with the AI systems as users, experts in the loop, or other decision makers.…”

Section: Valuesmentioning

confidence: 99%

“…However, such models can exacerbate statistical biases because restrictive assumptions on the training data often do not hold with nuanced demographics. Furthermore, designers who must make decisions on what variables to include or exclude can impart their own cognitive biases into the model [112,184]. Complex models are often used on nonlinear, multimodal data such as text and images.…”

Section: Algorithmic Effectsmentioning

confidence: 99%

See 1 more Smart Citation

Towards a standard for identifying and managing bias in artificial intelligence

Schwartz¹,

Vassilev²,

Greene³

et al. 2022

179

View full text Add to dashboard Cite

Bias is neither new nor unique to AI and it is not possible to achieve zero risk of bias in an AI system. NIST intends to develop methods for increasing assurance, GOVERNANCE and practice improvements for identifying, understanding, measuring, managing, and reducing bias. To reach this goal, techniques are needed that are flexible, can be applied across contexts regardless of industry, and are easily communicated to different stakeholder groups. To contribute to the growth of this burgeoning topic area, NIST will continue its work in measuring and evaluating computational biases, and seeks to create a hub for evaluating socio-technical factors. This will include development of formal guidance and standards, supporting standards development activities such as workshops and public comment periods for draft documents, and ongoing discussion of these topics with the stakeholder community.

show abstract

Section: Valuesmentioning

confidence: 99%

Section: Algorithmic Effectsmentioning

confidence: 99%

Towards a standard for identifying and managing bias in artificial intelligence

Schwartz¹,

Vassilev²,

Greene³

et al. 2022

179

View full text Add to dashboard Cite

show abstract

“…Examples of values that can be designed include "data privacy, accessibility, responsibility, accountability, transparency, explainability, efficiency, consent, inclusivity, diversity, security, and control" (Hasselbalch, 2021). (Also see Slota, et al, 2021;Umbrello & Van de Poel, 2021). Different stakeholders, ranging from developers to users, institutions or commercial entities, have different (often conflicting) data interests (Delgado, et al, 2021).…”

Section: Data Interestsmentioning

confidence: 99%

Learning analytics as data ecology: a tentative proposal

2023

View full text Add to dashboard Cite

Central to the institutionalization of learning analytics is the need to understand and improve student learning. Frameworks guiding the implementation of learning analytics flow from and perpetuate specific understandings of learning. Crucially, they also provide insights into how learning analytics acknowledges and positions itself as entangled in institutional data ecosystems, and (increasingly) as part of a data ecology driven by a variety of data interests. The success of learning analytics should therefore be understood in terms of data flows and data interests informing the emerging and mutually constitutive interrelationships and interdependencies between different stakeholders, interests and power relations. This article analyses several selected frameworks to determine the extent to which learning analytics understands itself as a data ecosystem with dynamic interdependencies and interrelationships (human and non-human). Secondly, as learning analytics increasingly becomes part of broader data ecologies, we examine the extent to which learning analytics takes cognizance of the reality, the potential and the risks of being part of a broader data ecology. Finally, this article examines the different data interests vested in learning analytics and critically considers implications for student data sovereignty. The research found that most of the analyzed frameworks understand learning analytics as a data ecosystem, with very little evidence of a broader data ecological understanding. The vast majority of analyzed frameworks consider student data as valuable resource without considering student data ownership and their data rights for self-determination.

show abstract

“…Unsurprisingly, ethical principles of AI are a major theme for many of the authors whose work appears here. For example, Slota et al (2021) conducted interviews with 26 stakeholders to explore the challenges of AI, including the distribution of agent empowerment and the difficulty of creating accountable systems. They propose the creation of accountable sociotechnical systems (cf.…”

Section: The Ethical Principles Of Aimentioning

confidence: 99%