The dangers of faulty, biased, or malicious algorithms requires independent oversight

Shneiderman, Ben

doi:10.1073/pnas.1618211113

Cited by 66 publications

(52 citation statements)

References 14 publications

(10 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is a long history of discussions to define trust such as Fukuyama (1995), who focused on social trust "within a community of regular, honest, and cooperative behavior, based on commonly shared norms, on the part of the members of that community." His community-based definition leads to raising trust by relying on respected independent oversight structures (Shneiderman, 2000(Shneiderman, , 2016 including:…”

Section: Reliable Safe and Trustworthy Systemsmentioning

confidence: 99%

Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy

Shneiderman

2020

International Journal of Human–Computer Interaction

Self Cite

527

211

View full text Add to dashboard Cite

Well-designed technologies that offer high levels of human control and high levels of computer automation can increase human performance, leading to wider adoption. The Human-Centered Artificial Intelligence (HCAI) framework clarifies how to (1) design for high levels of human control and high levels of computer automation so as to increase human performance, (2) understand the situations in which full human control or full computer control are necessary, and (3) avoid the dangers of excessive human control or excessive computer control. The methods of HCAI are more likely to produce designs that are Reliable, Safe & Trustworthy (RST). Achieving these goals will dramatically increase human performance, while supporting human self-efficacy, mastery, creativity, and responsibility.

show abstract

Section: Reliable Safe and Trustworthy Systemsmentioning

confidence: 99%

Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy

Shneiderman

2020

International Journal of Human–Computer Interaction

Self Cite

527

211

View full text Add to dashboard Cite

show abstract

“…However, there are equally dire predictions of out-of-control robots, biased decision making, unfair treatment of minority groups, privacy violations, adversarial attacks, and challenges to human rights. While the AI research community is shifting to emphasize Human-centered Artificial Intelligence (HCAI), there is also resistance to change [101].…”

Section: Introductionmentioning

confidence: 99%

Bridging the Gap Between Ethics and Practice

Shneiderman

2020

ACM Trans. Interact. Intell. Syst.

Self Cite

333

View full text Add to dashboard Cite

This article attempts to bridge the gap between widely discussed ethical principles of Human-centered AI (HCAI) and practical steps for effective governance. Since HCAI systems are developed and implemented in multiple organizational structures, I propose 15 recommendations at three levels of governance: team, organization, and industry. The recommendations are intended to increase the reliability, safety, and trustworthiness of HCAI systems: (1) reliable systems based on sound software engineering practices, (2) safety culture through business management strategies, and (3) trustworthy certification by independent oversight. Software engineering practices within teams include audit trails to enable analysis of failures, software engineering workflows, verification and validation testing, bias testing to enhance fairness, and explainable user interfaces. The safety culture within organizations comes from management strategies that include leadership commitment to safety, hiring and training oriented to safety, extensive reporting of failures and near misses, internal review boards for problems and future plans, and alignment with industry standard practices. The trustworthiness certification comes from industry-wide efforts that include government interventions and regulation, accounting firms conducting external audits, insurance companies compensating for failures, nongovernmental and civil society organizations advancing design principles, and professional organizations and research institutes developing standards, policies, and novel ideas. The larger goal of effective governance is to limit the dangers and increase the benefits of HCAI to individuals, organizations, and society. CCS Concepts: • Computing methodologies → Philosophical/theoretical foundations of artificial intelligence; • Human-centered computing → HCI theory, concepts and models; • Software and its engineering → Software development process management;

show abstract

“…The 15 recommendations are based on the HCAI framework and the combined designs from emulation and application research (Shneiderman, 2020c). These recommendations suggest how to: (1) adapt proven software engineering team practices,(2) implement organization-wide management strategies to build a safety culture, and (3) establish independent oversight methods (Shneiderman, 2016) that can be applied industry-wide to improve performance ( Figure 9). These new strategies guide software team leaders, business managers, and organization leaders in the methods to develop HCAI products and services that are driven by three goals:…”

Section: Governance Structures For Hcaimentioning

confidence: 99%

Human-Centered Artificial Intelligence: Three Fresh Ideas

Shneiderman¹

2020

THCI

Self Cite

195

View full text Add to dashboard Cite

Human-Centered AI (HCAI) is a promising direction for designing AI systems that support human self-efficacy, promote creativity, clarify responsibility, and facilitate social participation. These human aspirations also encourage consideration of privacy, security, environmental protection, social justice, and human rights. This commentary reverses the current emphasis on algorithms and AI methods, by putting humans at the center of systems design thinking, in effect, a second Copernican Revolution. It offers three ideas: (1) a two-dimensional HCAI framework, which shows how it is possible to have both high levels of human control AND high levels of automation, (2) a shift from emulating humans to empowering people with a plea to shift language, imagery, and metaphors away from portrayals of intelligent autonomous teammates towards descriptions of powerful tool-like appliances and tele-operated devices, and (3) a three-level governance structure that describes how software engineering teams can develop more reliable systems, how managers can emphasize a safety culture across an organization, and how industry-wide certification can promote trustworthy HCAI systems. These ideas will be challenged by some, refined by others, extended to accommodate new technologies, and validated with quantitative and qualitative research. They offer a reframe-a chance to restart design discussions for products and services-which could bring greater benefits to individuals, families, communities, businesses, and society.

show abstract

The dangers of faulty, biased, or malicious algorithms requires independent oversight

Cited by 66 publications

References 14 publications

Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy

Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy

Bridging the Gap Between Ethics and Practice

Human-Centered Artificial Intelligence: Three Fresh Ideas

Contact Info

Product

Resources

About