Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance

Raji, Inioluwa Deborah; Xu, Peggy; Honigsberg, Colleen; Ho, Daniel E.

doi:10.1145/3514094.3534181

Cited by 41 publications

(31 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The size of the system and its components also determine auditability; the datasets that large generative AI systems are trained on are not only difficult to analyze at scale, but few tools exist to analyze large static datasets [19]. Formal audits alone cannot be the only insight or governance of a system [76].…”

Section: Auditabilitymentioning

confidence: 99%

“…Critically, actors in this space must have some incentive to engage in frequent community discussion and be held accountable to commitments for safe releases. Google's public position on responsible AI practices encourages in-house risk evaluation and mitigation, but conflicts of interest can result in internal critics being unable to share or publish findings [76] and dismissal [30]. These initiatives can also be formed as an industry argument for self-regulation, but ultimately lack external accountability.…”

Section: Multidisciplinary Discoursementioning

confidence: 99%

See 1 more Smart Citation

The Gradient of Generative AI Release: Methods and Considerations

Solaiman¹

2023

Preprint

View full text Add to dashboard Cite

As increasingly powerful generative AI systems are developed, the release method greatly varies. We propose a framework to assess six levels of access to generative AI systems: fully closed; gradual or staged access; hosted access; cloud-based or API access; downloadable access; and fully open. Each level, from fully closed to fully open, can be viewed as an option along a gradient. We outline key considerations across this gradient: release methods come with tradeoffs, especially around the tension between concentrating power and mitigating risks. Diverse and multidisciplinary perspectives are needed to examine and mitigate risk in generative AI systems from conception to deployment. We show trends in generative system release over time, noting closedness among large companies for powerful systems and openness among organizations founded on principles of openness. We also enumerate safety controls and guardrails for generative systems and necessary investments to improve future releases.Preprint. Under review.

show abstract

Section: Auditabilitymentioning

confidence: 99%

Section: Multidisciplinary Discoursementioning

confidence: 99%

The Gradient of Generative AI Release: Methods and Considerations

Solaiman¹

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Disclosures seek to acknowledge the agency and autonomy of individuals. Calls for the requirement of disclosures in the context of AI systems appear in policy recommendations on algorithmic auditing (Costanza-Chock et al, 2022; Raji et al, 2022) and regulatory frameworks by the European Union (see articles 13 and 22 of the GDPR and articles 51, 52, and 60 of the AI Act), and US Congress (Klobuchar, 2018; Trahan, 2021). Meaningful disclosure affirms that the individual has final decision-making power in how they want to proceed.…”

Section: Dimension 5: Disclosure-centered Mediationmentioning

confidence: 99%

Terms-we-serve-with: Five dimensions for anticipating and repairing algorithmic harm

Rakova,

Shelby,

2023

Big Data & Society

View full text Add to dashboard Cite

Power and information asymmetries between people and digital technology companies are further legitimized through contractual agreements that fail to provide meaningful consent and contestability. In particular, the Terms-of-Service (ToS) agreement, is a contract of adhesion where companies effectively set the terms and conditions of the contract. Whereas, ToS reinforce existing structural inequalities, we seek to enable an intersectional accountability mechanism grounded in the practice of algorithmic reparation. Building on existing critiques of ToS in the context of algorithmic systems, we return to the roots of contract theory by recentering notions of agency and mutual assent. We evolve a multipronged intervention we frame as the Terms-we-Serve-with (TwSw) social, computational, and legal framework. The TwSw is a new social imaginary centered on: (1) co-constitution of user agreements, through participatory mechanisms; (2) addressing friction, leveraging the fields of design justice and critical design in the production and resolution of conflict; (3) enabling refusal mechanisms, reflecting the need for a sufficient level of human oversight and agency including opting out; (4) complaint and algorithmic harms reporting, through a feminist studies lens and open-sourced computational tools; and (5) disclosure-centered mediation, to disclose, acknowledge, and take responsibility for harm, drawing on the field of medical law. We further inform our analysis through an exploratory design workshop with a South African gender-based violence reporting AI startup. We derive practical strategies for communities, technologists, and policy-makers to leverage a relational approach to algorithmic reparation and propose there's a need for a radical restructuring of the “take-it-or-leave-it” ToS agreement.

show abstract

“…External model audit, i.e. model evaluation by an independent, external auditor for the purpose of providing a judgement -or input to a judgement -about the safety of deploying a model (or training a new one) (ARC Evals, 2023; Mökander et al, 2023;Raji et al, 2022b).…”

Section: Model Evaluation As Critical Governance Infrastructurementioning

confidence: 99%

“…(a) Surface emerging model behaviours and risks via monitoring efforts. This could include direct monitoring of inputs and outputs to the model, and systems for incident reporting (see Brundage et al, 2022;Raji et al, 2022b).…”

Section: Responsible Deploymentmentioning

confidence: 99%

Structured Access

Shevlane

2022

The Oxford Handbook of AI Governance

View full text Add to dashboard Cite

Structured access is an emerging paradigm for the safe deployment of artificial intelligence (AI). Instead of openly disseminating AI systems, developers facilitate controlled, arm’s length interactions with their AI systems. The aim is to prevent dangerous AI capabilities from being widely accessible, whilst preserving access to AI capabilities that can be used safely. The developer must both restrict how the AI system can be used and prevent the user from circumventing these restrictions through modification or reverse engineering of the AI system. Structured access is most effective when implemented through cloud-based AI services, rather than disseminating AI software that runs locally on users’ hardware. Cloud-based interfaces provide the AI developer greater scope for controlling how the AI system is used, and for protecting against unauthorized modifications to the system’s design. This chapter expands the discussion of “publication norms” in the AI community, which to date has focused on the question of how the informational content of AI research projects should be disseminated (e.g., code and models). Although this is an important question, there are limits to what can be achieved through the control of information flows. Structured access views AI software not only as information that can be shared but also as a tool with which users can have arm’s length interactions. There are early examples of structured access being practiced by AI developers, but there is much room for further development, both in the functionality of cloud-based interfaces and in the wider institutional framework.

show abstract

Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance

Cited by 41 publications

References 66 publications

The Gradient of Generative AI Release: Methods and Considerations

The Gradient of Generative AI Release: Methods and Considerations

Terms-we-serve-with: Five dimensions for anticipating and repairing algorithmic harm

Structured Access

Contact Info

Product

Resources

About