Proving Public Law Error in Automated Decision-Making Systems

Tomlinson, Joe; Sheridan, Katy; Harkens, Adam

doi:10.2139/ssrn.3476657

Cited by 3 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Importantly, identifying bias in the underlying code or data set of automated processes is difficult due to the opacity of ADM. Of course, administrative decisions made by humans can also be opaque, yet the types of opacity that may arise in ADM are 'distinct and more complex' in a number of ways. 117 Due to their black-box nature, the internal decision-making logic and mechanism of automated systems, and the choices made in selecting the data and programming the system, are generally hidden. 118 As Burrell notes, opacity can arise from the deliberate concealment of an algorithm for reasons such as protecting trade secrets or maintaining competitive advantage.…”

Section: Adm and New Bias Challengesmentioning

confidence: 99%

Addressing Disconnection: Automated Decision-Making, Administrative Law and Regulatory Reform

Huggins¹

2021

UNSWLJ

View full text Add to dashboard Cite

Automation is transforming how government agencies make decisions. This article analyses three distinctive features of automated decision-making that are difficult to reconcile with key doctrines of administrative law developed for a human-centric decision-making context. First, the complex, multi-faceted decision-making requirements arising from statutory interpretation and administrative law principles raise questions about the feasibility of designing automated systems to cohere with these expectations. Secondly, whilst the courts have emphasised a human mental process as a criterion of a valid decision, many automated decisions are made with limited or no human input. Thirdly, the new types of bias associated with opaque automated decision-making are not easily accommodated by the bias rule, or other relevant grounds of judicial review. This article, therefore, argues that doctrinal and regulatory evolution are both needed to address these disconnections and maintain the accountability and contestability of administrative decisions in the digital age.

show abstract

Section: Adm and New Bias Challengesmentioning

confidence: 99%

Addressing Disconnection: Automated Decision-Making, Administrative Law and Regulatory Reform

Huggins¹

2021

UNSWLJ

View full text Add to dashboard Cite

show abstract

“…A second issue in relation to legality is the inherent opacity of algorithms. Many algorithms are intentionally designed to be opaque to protect competitive commercial advantage, or are intrinsically 'illiterate' so that only those with technical expertise can understand them (Tomlinson et al 2019). Indeed, as systems become more sophisticated and 'begin to learn, iterate, and improve upon themselves in unpredictable or otherwise unintelligible ways, their logic often becomes less intuitive to human onlookers' (Pasquale 2015; Bots at the Gate 2018, P. 11).…”

Section: Automated Decision-makingmentioning

confidence: 99%

“…The opacity inherent to algorithms will inevitably impact the way evidence is considered in judicial review. Tomlinson et al (2019) argue that there will likely be a greater emphasis on certain types of evidence, such as documents produced in the course of the algorithmic design process. New investigative techniques by lawyers, such as audits, may be needed, and there will be an increased reliance on expert opinion.…”

Section: Automated Decision-makingmentioning

confidence: 99%

See 1 more Smart Citation

Technology, Displaced? The Risks and Potential of Artificial Intelligence for Fair, Effective, and Efficient Refugee Status Determination

Kinchin

2021

LiC

View full text Add to dashboard Cite

Human vulnerability is at the core of refugee status determination and human rights provides its regulatory frame, so to speak of artificial intelligence within the refugee context may seem troubling at least, dystopian at worst. But the rapid development of artificial intelligence in government decision-making will unlikely be slowed by such ethical quandaries. The potential integration of automation, machine learning and algorithmic decision-making into global migration regulation and policy has far-reaching implications for refugee law. The consequences for efficiency, legality, accountability, transparency and human rights warrant a timely and critical conversation about the possible impact of existing and future technologies on refugee status determination. Predictive analytics, biometrics, automated credibility assessments and algorithmic decision-making are technologies that could have utility for refugee status determination processing, credibility assessments and decision-making. Each technology is considered through a lens of ‘risk and potential’, which is measured in terms of ‘fair, efficient and effective’ refugee status determination. The opportunities that artificial intelligence offers for efficiency and effectiveness in refugee status determination are compelling. Artificial intelligence allows for faster data processing and the ability to undertake high-volume, repetitive tasks. Increased consistency and up-to-date information, a capacity to plan for workloads and predict movements and the potential to ‘design out’ existing biases, promise to deliver positive outcomes for asylum seekers. But the risks of integrating artificial intelligence in a decision-making process that is defined by human vulnerability loom large. The lack of transparency in algorithms may result in a denial of procedural fairness, and algorithmic bias continues to be a vexing issue. If refugee and human rights are denied, international protection may be compromised. Technical and contextual issues may increase the potential for error, and unanswered questions remain around legality.

show abstract

“…Others have also applied concepts from administrative law (English or otherwise) to ADM in various ways [11,15,46,55,70,78]. However, these have generally not viewed ADM as a broad process, or, as with our previous work, sought to apply legal standards to public sector ADM. We believe that administrative law's understanding of decision-making as a process is also highly relevant to accountability of ADM and algorithmic systems more generallyin private and public sectors.…”

Section: Introductionmentioning

confidence: 99%

Reviewable Automated Decision-Making

Cobbe

Lee

Singh

2021

Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency

View full text Add to dashboard Cite

This paper introduces reviewability as a framework for improving the accountability of automated and algorithmic decisionmaking (ADM) involving machine learning. We draw on an understanding of ADM as a socio-technical process involving both human and technical elements, beginning before a decision is made and extending beyond the decision itself. While explanations and other model-centric mechanisms may assist some accountability concerns, they often provide insufficient information of these broader ADM processes for regulatory oversight and assessments of legal compliance. Reviewability involves breaking down the ADM process into technical and organisational elements to provide a systematic framework for determining the contextually appropriate record-keeping mechanisms to facilitate meaningful review-both of individual decisions and of the process as a whole. We argue that a reviewability framework, drawing on administrative law's approach to reviewing human decision-making, offers a practical way forward towards more a more holistic and legally-relevant form of accountability for ADM.

show abstract

Proving Public Law Error in Automated Decision-Making Systems

Cited by 3 publications

References 0 publications

Addressing Disconnection: Automated Decision-Making, Administrative Law and Regulatory Reform

Addressing Disconnection: Automated Decision-Making, Administrative Law and Regulatory Reform

Technology, Displaced? The Risks and Potential of Artificial Intelligence for Fair, Effective, and Efficient Refugee Status Determination

Reviewable Automated Decision-Making

Contact Info

Product

Resources

About