This article addresses the problem of verifying the safety of autonomous systems with neural network (NN) controllers. We focus on NNs with sigmoid/tanh activations and use the fact that the sigmoid/tanh is the solution to a quadratic differential equation. This allows us to convert the NN into an equivalent hybrid system and cast the problem as a hybrid system verification problem, which can be solved by existing tools. Furthermore, we improve the scalability of the proposed method by approximating the sigmoid with a Taylor series with worst-case error bounds. Finally, we provide an evaluation over four benchmarks, including comparisons with alternative approaches based on mixed integer linear programming as well as on star sets.

This paper addresses the problem of detection and identification of sensor attacks in the presence of transient faults. We consider a system with multiple sensors measuring the same physical variable, where some sensors might be under attack and provide malicious values. We consider a setup, in which each sensor provides the controller with an interval of possible values for the true value. While approaches exist for detecting malicious sensor attacks, they are conservative in that they treat attacks and faults in the same way, thus neglecting the fact that sensors may provide faulty measurements at times due to temporary disturbances (e.g., a tunnel for GPS). To address this problem, we propose a transient fault model for each sensor and an algorithm designed to detect and identify attacks in the presence of transient faults. The fault model consists of three aspects: the size of the sensor's interval (1) and an upper bound on the number of errors (2) allowed in a given window size (3). Given such a model for each sensor, the algorithm uses pairwise inconsistencies between sensors to detect and identify attacks. In addition to the algorithm, we provide a framework for selecting a fault model for each sensor based on training data. Finally, we validate the algorithm's performance on real measurement data obtained from an unmanned ground vehicle.

This paper presents Verisig 2.0, a verification tool for closed-loop systems with neural network (NN) controllers. We focus on NNs with tanh/sigmoid activations and develop a Taylor-model-based reachability algorithm through Taylor model preconditioning and shrink wrapping. Furthermore, we provide a parallelized implementation that allows Verisig 2.0 to efficiently handle larger NNs than existing tools can. We provide an extensive evaluation over 10 benchmarks and compare Verisig 2.0 against three state-of-the-art verification tools. We show that Verisig 2.0 is both more accurate and faster, achieving speed-ups of up to 21x and 268x against different tools, respectively.

Fault diagnosis in networked systems has been an extensively studied field in systems engineering. Fault diagnosis generally includes the tasks of fault detection and isolation, and optionally recovery (FDIR). In this paper we further consider the blame assignment problem: given a system trace on which a system failure occurred and an identified set of faulty components, determine which subsets of faulty components are the culprits for the system failure.We provide formal definitions of the notion culprits and the blame assignment problem, under the assumptions that only one system trace is given and the system cannot be rerun. We show that the problem is equivalent to deciding the unsatisfiability of a set of logical constraints on component behaviors, and present the transformation from a blame assignment instance into an instance of unsatisfiability checking. We also apply the approach to a case study in the medical device interoperability scenario that has motivated our work. ABSTRACTFault diagnosis in networked systems has been an extensively studied field in systems engineering. Fault diagnosis generally includes the tasks of fault detection and isolation, and optionally recovery (FDIR). In this paper we further consider the blame assignment problem: given a system trace on which a system failure occurred and an identified set of faulty components, determine which subsets of faulty components are the culprits for the system failure.We provide formal definitions of the notion culprits and the blame assignment problem, under the assumptions that only one system trace is given and the system cannot be rerun. We show that the problem is equivalent to deciding the unsatisfiability of a set of logical constraints on component behaviors, and present the transformation from a blame assignment instance into an instance of unsatisfiability checking. We also apply the approach to a case study in the medical device interoperability scenario that has motivated our work.

A logger in the cloud capable of keeping a secure, time-synchronized and tamper-evident log of medical device and patient information allows efficient forensic analysis in cases of adverse events or attacks on interoperable medical devices. A secure logger as such must meet requirements of confidentiality and integrity of message logs and provide tamper-detection and tamper-evidence. In this paper, we propose a design for such a cloud-based secure logger using the Intel Software Guard Extensions (SGX) and the Trusted Platform Module (TPM). The proposed logger receives medical device information from a dongle attached to a medical device. The logger relies on SGX, TPM and standard encryption to maintain a secure communication channel even on an untrusted network and operating system. We also show that the logger is resilient against different kinds of attacks such as Replay attacks, Injection attacks and Eavesdropping attacks. Abstract-A logger in the cloud capable of keeping a secure, time-synchronized and tamper-evident log of medical device and patient information allows efficient forensic analysis in cases of adverse events or attacks on interoperable medical devices. A secure logger as such must meet requirements of confidentiality and integrity of message logs and provide tamper-detection and tamper-evidence. In this paper, we propose a design for such a cloud-based secure logger using the Intel Software Guard Extensions (SGX) and the Trusted Platform Module (TPM). The proposed logger receives medical device information from a dongle attached to a medical device. The logger relies on SGX, TPM and standard encryption to maintain a secure communication channel even on an untrusted network and operating system. We also show that the logger is resilient against different kinds of attacks such as Replay attacks, Injection attacks and Eavesdropping attacks.

This paper aims to improve the design of modern Medical Cyber Physical Systems through the addition of supplemental noninvasive monitors. Specifically, we focus on monitoring the arterial blood oxygen content (C a O 2 ), one of the most closely observed vital signs in operating rooms, currently measured by a proxyperipheral hemoglobin oxygen saturation (S p O 2 ). While S p O 2 is a good estimate of O 2 content in the finger where it is measured, it is a delayed measure of its content in the arteries. In addition, it does not incorporate system dynamics and is a poor predictor of future C a O 2 values. Therefore, as a first step towards supplementing the usage of S p O 2 , this work introduces a predictive monitor designed to provide early detection of critical drops in C a O 2 caused by a pulmonary shunt in infants.To this end, we develop a formal model of the circulation of oxygen and carbon dioxide in the body, characterized by unknown patient-unique parameters. Employing the model, we design a matched subspace detector to provide a near constant false alarm rate invariant to these parameters and modeling uncertainties. Finally, we validate our approach on real-patient data from lung lobectomy surgeries performed at the Children's Hospital of Philadelphia. Given 198 infants, the detector predicted 81% of the critical drops in C a O 2 at an average of about 65 seconds earlier than the S p O 2 -based monitor, while achieving a 0:9% false alarm rate (representing about 2 false alarms per hour). ABSTRACTThis paper aims to improve the design of modern Medical Cyber Physical Systems through the addition of supplemental noninvasive monitors. Specifically, we focus on monitoring the arterial blood oxygen content (CaO2), one of the most closely observed vital signs in operating rooms, currently measured by a proxy -peripheral hemoglobin oxygen saturation (SpO2). While SpO2 is a good estimate of O2 content in the finger where it is measured, it is a delayed measure of its content in the arteries. In addition, it does not incorporate system dynamics and is a poor predictor of future CaO2 values. Therefore, as a first step towards supplementing the usage of SpO2, this work introduces a predictive monitor designed to provide early detection of critical drops in CaO2 caused by a pulmonary shunt in infants.To this end, we develop a formal model of the circulation of oxygen and carbon dioxide in the body, characterized by unknown patient-unique parameters. Employing the model, we design a matched subspace detector to provide a near constant false alarm rate invariant to these parameters and modeling uncertainties. Finally, we validate our approach on real-patient data from lung lobectomy surgeries performed at the Children's Hospital of Philadelphia. Given 198 infants, the detector predicted 81% of the critical drops in CaO2 at an average of about 65 seconds earlier than the SpO2-based monitor, while achieving a 0.9% false alarm rate (representing about 2 false alarms per hour).

No abstract

The tight interaction between information technology and the physical world inherent in Cyber-Physical Systems (CPS) can challenge traditional approaches for monitoring safety and security. Data collected for robust CPS monitoring is often sparse and may lack rich training data describing critical events/attacks. Moreover, CPS often operate in diverse environments that can have significant inter/intra-system variability. Furthermore, CPS monitors that are not robust to data sparsity and inter/intra-system variability may result in inconsistent performance and may not be trusted for monitoring safety and security. Towards overcoming these challenges, this paper presents recent work on the design of parameter-invariant (PAIN) monitors for CPS. PAIN monitors are designed such that unknown events and system variability minimally affect the monitor performance. This work describes how PAIN designs can achieve a constant false alarm rate (CFAR) in the presence of data sparsity and intra/inter system variance in real-world CPS. To demonstrate the design of PAIN monitors for safety monitoring in CPS with different types of dynamics, we consider systems with networked dynamics, linear-time invariant dynamics, and hybrid dynamics that are discussed through case studies for building actuator fault detection, meal detection in type I diabetes, and detecting hypoxia caused by pulmonary shunts in infants. In all applications, the PAIN monitor is shown to have (significantly) less variance in monitoring performance and (often) outperforms other competing approaches in the literature. Finally, an initial application of PAIN monitoring for CPS security is presented along with challenges and research directions for future security monitoring deployments. Disciplines Computer Engineering | Computer Sciences Comments