Software product quality assurance

Ryan, John R.

doi:10.1145/1500774.1500823

Cited by 4 publications

(1 citation statement)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We define algorithm auditing as ‘the research and practice of assessing, mitigating, and assuring an algorithm’s legality, ethics and safety’. This algorithm auditing work draws from other areas, such as ethics, law, software security and certification ( Hagendorff 2020 ; Jobin et al 2019 ; Kazim et al 2021 ; Pasquale 2019 ; Robertson et al 2018 ; Rushby 1988 ; Ryan 1982 ; Shneiderman 2016 ; Voas and Miller 2006 ). As will become clear in the following, we concentrate on mechanisms of assessing the performance of a system through the perspective of an engineering-focused technical audit.…”

Section: Assessing Ai-driven Recruitment Systems Through Auditmentioning

confidence: 99%

Systematizing Audit in Algorithmic Recruitment

et al. 2021

View full text Add to dashboard Cite

Business psychologists study and assess relevant individual differences, such as intelligence and personality, in the context of work. Such studies have informed the development of artificial intelligence systems (AI) designed to measure individual differences. This has been capitalized on by companies who have developed AI-driven recruitment solutions that include aggregation of appropriate candidates (Hiretual), interviewing through a chatbot (Paradox), video interview assessment (MyInterview), and CV-analysis (Textio), as well as estimation of psychometric characteristics through image-(Traitify) and game-based assessments (HireVue) and video interviews (Cammio). However, driven by concern that such high-impact technology must be used responsibly due to the potential for unfair hiring to result from the algorithms used by these tools, there is an active effort towards proving mechanisms of governance for such automation. In this article, we apply a systematic algorithm audit framework in the context of the ethically critical industry of algorithmic recruitment systems, exploring how audit assessments on AI-driven systems can be used to assure that such systems are being responsibly deployed in a fair and well-governed manner. We outline sources of risk for the use of algorithmic hiring tools, suggest the most appropriate opportunities for audits to take place, recommend ways to measure bias in algorithms, and discuss the transparency of algorithms.

show abstract

Section: Assessing Ai-driven Recruitment Systems Through Auditmentioning

confidence: 99%

Systematizing Audit in Algorithmic Recruitment

et al. 2021

View full text Add to dashboard Cite

show abstract

Machine Learning for Health: Algorithm Auditing & Quality Control

et al. 2021

View full text Add to dashboard Cite

Developers proposing new machine learning for health (ML4H) tools often pledge to match or even surpass the performance of existing tools, yet the reality is usually more complicated. Reliable deployment of ML4H to the real world is challenging as examples from diabetic retinopathy or Covid-19 screening show. We envision an integrated framework of algorithm auditing and quality control that provides a path towards the effective and reliable application of ML systems in healthcare. In this editorial, we give a summary of ongoing work towards that vision and announce a call for participation to the special issue Machine Learning for Health: Algorithm Auditing & Quality Control in this journal to advance the practice of ML4H auditing.

show abstract

Algorithm Auditing: Managing the Legal, Ethical, and Technological Risks of Artificial Intelligence, Machine Learning, and Associated Algorithms

2022

View full text Add to dashboard Cite

Business reliance on algorithms are becoming ubiquitous, and companies are increasingly concerned about their algorithms causing major financial or reputational damage. High-profile cases include VW's Dieselgate scandal with fines worth of $34.69B, Knight Capital's bankruptcy (~$450M) by a glitch in its algorithmic trading system, and Amazon's AI recruiting tool being scrapped after showing bias against women. In response, governments are legislating and imposing bans, regulators fining companies, and the Judiciary discussing potentially making algorithms artificial "persons" in Law. Soon there will be 'billions' of algorithms making decisions with minimal human intervention; from autonomous vehicles and finance, to medical treatment, employment, and legal decisions. Indeed, scaling to problems beyond the human is a major point of using such algorithms in the first place. As with Financial Audit, governments, business and society will require Algorithm Audit; formal assurance that algorithms are legal, ethical and safe. A new industry is envisaged: Auditing and Assurance of Algorithms (cf. Data privacy), with the remit to professionalize and industrialize AI, ML and associated algorithms.The stakeholders range from those working on policy and regulation, to industry practitioners and developers. We also anticipate the nature and scope of the auditing levels and framework presented will inform those interested in systems of governance and compliance to regulation/standards. Our goal in this paper is to survey the key areas necessary to perform auditing and assurance, and instigate the debate in this novel area of research and practice.

show abstract

Software product quality assurance

Cited by 4 publications

References 1 publication

Systematizing Audit in Algorithmic Recruitment

Systematizing Audit in Algorithmic Recruitment

Machine Learning for Health: Algorithm Auditing & Quality Control

Algorithm Auditing: Managing the Legal, Ethical, and Technological Risks of Artificial Intelligence, Machine Learning, and Associated Algorithms

Contact Info

Product

Resources

About