The myths and costs of autonomous weapon systems

Hoffman, Robert R.; Cullen, Timothy; Hawley, John K.

doi:10.1080/00963402.2016.1194619

Cited by 4 publications

(3 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The U.S. Defense Science Board (Murphy and Shields, 2012) report describes the ten levels of automation/autonomy this way: "though attractive, the conceptualization of levels of autonomy as a scientific grounding for a developmental roadmap has been unproductive." A later report described many opportunities and dangers (Defense Science Board 2016), while cognitive science researchers (Hoffman et al, 2016) point to the failures and costs of autonomous weapons.…”

mentioning

confidence: 99%

Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy

Shneiderman

2020

International Journal of Human–Computer Interaction

518

211

View full text Add to dashboard Cite

Well-designed technologies that offer high levels of human control and high levels of computer automation can increase human performance, leading to wider adoption. The Human-Centered Artificial Intelligence (HCAI) framework clarifies how to (1) design for high levels of human control and high levels of computer automation so as to increase human performance, (2) understand the situations in which full human control or full computer control are necessary, and (3) avoid the dangers of excessive human control or excessive computer control. The methods of HCAI are more likely to produce designs that are Reliable, Safe & Trustworthy (RST). Achieving these goals will dramatically increase human performance, while supporting human self-efficacy, mastery, creativity, and responsibility.

show abstract

mentioning

confidence: 99%

Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy

Shneiderman

2020

International Journal of Human–Computer Interaction

518

211

View full text Add to dashboard Cite

show abstract

“…Hoffman et al (2016) highlighted hazards of autonomous weapons systems that make engagement decisions independent of their human supervisors. Those systems have attacked friendly forces, resulting in unintended fatalities.…”

Section: A Contemporary Safety Riskmentioning

confidence: 99%

John Senders, Human Error, and System Safety

Strauch¹

2021

Hum Factors

View full text Add to dashboard Cite

Objective I examine John Senders’ work and discuss his influence on the study of error causation,error mitigation, and sociotechnical system safety. Background John Senders’ passing calls for an evaluation of the impact of his work. Method I review literature and accident investigation findings to discuss themes in Senders’ work and potential associations between that work and error causation and system safety. Results Senders consistently emphasized empirical rigor and theoretical exploration in his research, with the desire to apply that work to enhance human performance. He has contributed to changing the way error has been viewed, and to developing and implementing programs and techniques to mitigate error. While a causal relationship between Senders’ work and safety cannot be established, an association can be drawn between his research and efforts to mitigate error. Conclusion Because of Senders’ work, we have a better understanding of error causation and enhanced ways of mitigating system errors. However, new sources of error, involving advanced systems and operators’ knowledge and understanding of their functionalities can, if not addressed, degrade system safety. Application Modifications to advanced automation and operator training are suggested, and research to improve operator expertise in interacting with automated systems proposed.

show abstract

“…A regular United Nations Convention on Certain Conventional Weapons in Geneva attracts representatives of 125 countries who are drafting a treaty restricting the use of LAWS. Their case is bolstered by reports from cognitive science researchers [19], who document the failures, dangers, and costs of autonomous weapons. However, some military leaders do not wish to be limited, when they fear that adversaries will adopt autonomous weapons.…”

Section: Autonomous System or Supervisory Control?mentioning

confidence: 99%

Design Lessons From AI’s Two Grand Goals: Human Emulation and Useful Applications

Shneiderman

2020

IEEE Trans. Technol. Soc.

View full text Add to dashboard Cite

Researchers' goals shape the questions they raise, collaborators they choose, methods they use, and outcomes of their work. This article offers a fresh vision of artificial intelligence (AI) research by suggesting a simplification to two goals: 1) emulation to understand human abilities to build systems that perform tasks as well as or better than humans and 2) application of AI methods to build widely used products and services. Researchers and developers for each goal can fruitfully work along their desired paths, but this article is intended to limit the problems that arise when assumptions from one goal are used to drive work on the other goal. For example, autonomous humanoid robots are prominent with emulation researchers, but application developers avoid them, in favor of tool-like appliances or teleoperated devices for widely used commercial products and services. This article covers four such mismatches in goals that affect AI-guided application development: 1) intelligent agent or powerful tool; 2) simulated teammate or teleoperated device; 3) autonomous system or supervisory control; and 4) humanoid robot or mechanoid appliance. This article clarifies these mismatches to facilitate the discovery of workable compromise designs that will accelerate human-centered AI applications research. A greater emphasis on human-centered AI could reduce AI's existential threats and increase benefits for users and society, such as in business, education, healthcare, environmental preservation, and community safety.

show abstract

The myths and costs of autonomous weapon systems

Cited by 4 publications

References 9 publications

Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy

Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy

John Senders, Human Error, and System Safety

Design Lessons From AI’s Two Grand Goals: Human Emulation and Useful Applications

Contact Info

Product

Resources

About