Machine learning algorithms increasingly influence our decisions and interact with us in all parts of our daily lives. Therefore, just as we consider the safety of power plants, highways, and a variety of other engineered socio-technical systems, we must also take into account the safety of systems involving machine learning. Heretofore, the definition of safety has not been formalized in a machine learning context. In this article, we do so by defining machine learning safety in terms of risk, epistemic uncertainty, and the harm incurred by unwanted outcomes. We then use this definition to examine safety in all sorts of applications in cyber-physical systems, decision sciences, and data products. We find that the foundational principle of modern statistical machine learning, empirical risk minimization, is not always a sufficient objective. We discuss how four different categories of strategies for achieving safety in engineering, including inherently safe design, safety reserves, safe fail, and procedural safeguards can be mapped to a machine learning context. We then discuss example techniques that can be adopted in each category, such as considering interpretability and causality of predictive models, objective functions beyond expected prediction accuracy, human involvement for labeling difficult or rare examples, and user experience design of software and open data.
Cyber-physical systems for robotic surgery have enabled minimally invasive procedures with increased precision and shorter hospitalization. However, with increasing complexity and connectivity of software and major involvement of human operators in the supervision of surgical robots, there remain significant challenges in ensuring patient safety. This paper presents a safety monitoring system that, given the knowledge of the surgical task being performed by the surgeon, can detect safety-critical events in real-time. Our approach integrates a surgical gesture classifier that infers the operational context from the time-series kinematics data of the robot with a library of erroneous gesture classifiers that given a surgical gesture can detect unsafe events. Our experiments using data from two surgical platforms show that the proposed system can detect unsafe events caused by accidental or malicious faults within an average reaction time window of 1,693 milliseconds and F1 score of 0.88 and human errors within an average reaction time window of 57 milliseconds and F1 score of 0.76. I. INTRODUCTIONRobot-assisted surgery (RAS) is now a standard procedure across various surgical specialties, including gynecology, urology and general surgeries. During the last two decades, over 2 million procedures were performed using the Intuitive Surgical's daVinci robot in the U.S. [1]. Surgical robots are complex human-in-the-loop Cyber-Physical Systems (CPS) that enable 3D visualization of surgical field and more precise manipulation of surgical instruments such as scissors, graspers, and electro-cautery inside patient's body. The current generation of surgical robots are not fully autonomous yet. They are in level 0 of autonomy [2], following the commands provided by the surgeons from a master-side teleoperation console in real-time (Figure 1a), translating them into precise movements of robotic arms and instruments, while scaling surgeon's motions and filtering out handtremors. By increasing flexibility and precision, surgical robots have enabled new types of surgical procedures and have reduced complication rates and procedure times.Recent studies have shown that safety in robotic surgery may be compromised by vulnerabilities of the surgical robots to accidental or maliciously-crafted faults in the cyber or physical layers of the control system or human errors [3], [4]. Examples of system faults include disruptions of the communication between the surgeon console and the robot, causing packet drops or delays in tele-operation [5], accidental or malicious faults targeting the robot control software [6],
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
customersupport@researchsolutions.com
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.