Feeling fixes: Mess and emotion in algorithmic audits

Keyes, Os; Austin, Jeanie

doi:10.1177/20539517221113772

Cited by 7 publications

(3 citation statements)

References 77 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An algorithmic audit (Benjamin, 2019;Brown et al, 2021;Bucher, 2018;Burrell, 2016) uses experimental methods to analyze a piece of digital infrastructure such as a ranking algorithm on a website. Other applications of algorithmic audits include social media feeds (Bucher, 2018) facial recognition software (Banerjee et al, 2022;Keyes and Austin, 2022), or employment advertising programs (Imana et al, 2021). In situations where access to a given technology is limited, algorithmic audits offer a way to investigate algorithmic governance.…”

Section: Designing An Algorithmic Auditmentioning

confidence: 99%

Banking on alternative credit scores: Auditing the calculative infrastructure of U.S. consumer lending

McCanless

2023

Environ Plan A

View full text Add to dashboard Cite

Alternative credit scores have become an increasingly important tool for lenders to assess risk and authorize investment in consumer debt. Using alternative data and processing techniques that leverage machine learning (ML) and Artificial Intelligence (AI), these models are designed to bypass existing barriers to risk-based pricing, which is the idea that financial institutions offer different interest rates to consumers based on their likelihood of default. Through an algorithmic audit of one lender's (Upstart) credit scoring model, I find that alternative data, particularly whether an applicant has a bachelor's degree, strongly impacted loan outcomes. This raises important equity concerns about overhauling lending criteria via opaque models that restructure the logic of risk assessment. In following the logic of risk assessment generated by Upstart's model, I also audit three fintech-bank partnerships and examine the balance sheets of banks providing capital via Upstart's platform. This is done to demonstrate rising capital allocation to these types of loans at banks engaged in fintech-bank partnerships, in one case rising from 0.14% to 15.6% of the banks’ balance sheet over three years. My analysis shows that alternative credit scoring systems function as a key piece of calculative infrastructure, which allows some institutions to bypass barriers to risk-based pricing, and becomes an infrastructural site for tech startups to partner with financial institutions seeking out new sources of revenue.

show abstract

Section: Designing An Algorithmic Auditmentioning

confidence: 99%

Banking on alternative credit scores: Auditing the calculative infrastructure of U.S. consumer lending

McCanless

2023

Environ Plan A

View full text Add to dashboard Cite

show abstract

“…This situation can then be hard to correct, as simply removing a badly phrased question from a survey can lead to knock-on changes due (for instance) to priming. This echoes ethical issues with datasets with personal information used in computer science, which can continue being used despite major flaws being found [16].…”

Section: Introductionmentioning

confidence: 99%

Minimalist approaches to enforce privacy by design in surveys

Blanchard,

Gabasova

2023

RoCHI - International Conference on Human-Computer Interaction

View full text Add to dashboard Cite

Public institutions and private companies both frequently rely on user surveys for a variety of assessments (e.g. equality issues or quality of work environment). However, many such surveys struggle to garner sufficient responses, especially when they ask about sensitive subjects (such as work harassment), which also makes them exist in a legal grey area when it comes to data protection laws. One important factor in this issue is the perceived threat of deanonymisation, compounded by the frequent lack of transparency on how the data is used. The proposals seeking to address this issue often focus on complex cryptography (e.g. homomorphic encryption), without addressing the fears of non-technical users.This paper explores a radically different approach which minimises data collection on multiple fronts, partially by limiting the power of survey organisers. By design, it prevents generic attempts to deanonymise participants, as the server never stores even pseudonymised information. We also try to address questions of inclusivity, once again through a minimalist approach. Finally, we report on the first live test of a prototype developed following this approach.

show abstract

“…However, it has been highlighted in several studies that these responses are associated with their own set of challenges because iterations of datasets are still present even after the original set is removed (Peng, 2020). Moreover, the methods of tracing and accounting for harmful datasets, for instance through dataset auditing practices, comes with its own methodological challenges and ethical conundrums (Raji et al 2020; Keyes and Austin 2022). The challenges of hindering the reuse of contested data have also engendered new strategies to protect corporations against data as a ‘toxic asset’ (Schneier, 2016), and new discourses that frame sensitive data as potentially ‘toxic waste’ that can ‘overspill’ (Schwarzkopf, 2020).…”

Section: Introductionmentioning

confidence: 99%

Politics of data reuse in machine learning systems: Theorizing reuse entanglements

et al. 2022

View full text Add to dashboard Cite

Policy discussions and corporate strategies on machine learning are increasingly championing data reuse as a key element in digital transformations. These aspirations are often coupled with a focus on responsibility, ethics and transparency, as well as emergent forms of regulation that seek to set demands for corporate conduct and the protection of civic rights. And the Protective measures include methods of traceability and assessments of ‘good’ and ‘bad’ datasets and algorithms that are considered to be traceable, stable and contained. However, these ways of thinking about both technology and ethics obscure a fundamental issue, namely that machine learning systems entangle data, algorithms and more-than-human environments in ways that challenge a well-defined separation. This article investigates the fundamental fallacy of most data reuse strategies as well as their regulation and mitigation strategies that data can somehow be followed, contained and controlled in machine learning processes. Instead, the article argues that we need to understand the reuse of data as an inherently entangled phenomenon. To examine this tension between the discursive regimes and the realities of data reuse, we advance the notion of reuse entanglements as an analytical lens. The main contribution of the article is the conceptualization of reuse that places entanglements at its core and the articulation of its relevance using empirical illustrations. This is important, we argue, for our understanding of the nature of data and algorithms, for the practical uses of data and algorithms and our attitudes regarding ethics, responsibility and regulation.

show abstract

Feeling fixes: Mess and emotion in algorithmic audits

Cited by 7 publications

References 77 publications

Banking on alternative credit scores: Auditing the calculative infrastructure of U.S. consumer lending

Banking on alternative credit scores: Auditing the calculative infrastructure of U.S. consumer lending

Minimalist approaches to enforce privacy by design in surveys

Politics of data reuse in machine learning systems: Theorizing reuse entanglements

Contact Info

Product

Resources

About