ReSonAte: A Runtime Risk Assessment Framework for Autonomous Systems

Hartsell, Charles; Ramakrishna, Shreyas; Dubey, Abhishek; Stojcsics, Dániel; Mahadevan, Nagabhushan; Karsai, Gábor

doi:10.1109/seams51251.2021.00025

Cited by 17 publications

(17 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…3-c. Such an abrupt change in the image feature increases the AV's risk of collision as demonstrated in our recent work [26]. So, identifying the change points can be beneicial for reducing the system's risk of a consequence.…”

Section: Problem Formulationmentioning

confidence: 78%

“…In comparison, the detectors that used all the latent variables for detection had lower minimum sensitivity, as illustrated in the Table . Besides identifying if the input images are OOD to the training data, it is necessary to check if they have changed compared to the previous sequence of images in the time series. Such abrupt changes in the current input will increase the system's risk of consequence (e.g., collision) [26], so it is critical to detect them. his problem is necessary to be addressed but has not been given much importance in the OOD detection literature.…”

Section: Discussionmentioning

confidence: 99%

“…Identifying these changes is referred to as change point detection in literature. Change points in the values of a feature can increase the system's risk, as illustrated in our previous work [26]. So, it is critical to identify these change points during operation.…”

Section: Introductionmentioning

confidence: 92%

“…Finally, the outputs of the detector (See Fig. 6) are sent to the decision manager, which can use these detection results to perform system risk estimation [26] or high level controller selection using simplex strategies [64]. In this work, we use a simple decision logic that uses the OOD detection result to perform a control arbitration from the DNN controller to an autopilot controller.…”

Section: Runtime Out-of-distribution Detectionmentioning

confidence: 99%

See 3 more Smart Citations

Efficient Out-of-Distribution Detection Using Latent Space of β -VAE for Cyber-Physical Systems

Ramakrishna

Rahiminasab

Karsai

et al. 2022

ACM Trans. Cyber-Phys. Syst.

Self Cite

View full text Add to dashboard Cite

Deep Neural Networks are actively being used in the design of autonomous Cyber-Physical Systems (CPSs). The advantage of these models is their ability to handle high-dimensional state-space and learn compact surrogate representations of the operational state spaces. However, the problem is that the sampled observations used for training the model may never cover the entire state space of the physical environment, and as a result, the system will likely operate in conditions that do not belong to the training distribution. These conditions that do not belong to training distribution are referred to as Out-of-Distribution (OOD). Detecting OOD conditions at runtime is critical for the safety of CPS. In addition, it is also desirable to identify the context or the feature(s) that are the source of OOD to select an appropriate control action to mitigate the consequences that may arise because of the OOD condition. In this paper, we study this problem as a multi-labeled time series OOD detection problem over images, where the OOD is defined both sequentially across short time windows (change points) as well as across the training data distribution. A common approach to solving this problem is the use of multi-chained one-class classifiers. However, this approach is expensive for CPSs that have limited computational resources and require short inference times. Our contribution is an approach to design and train a single β -Variational Autoencoder detector with a partially disentangled latent space sensitive to variations in image features. We use the feature sensitive latent variables in the latent space to detect OOD images and identify the most likely feature(s) responsible for the OOD. We demonstrate our approach using an Autonomous Vehicle in the CARLA simulator and a real-world automotive dataset called nuImages.

show abstract

Section: Problem Formulationmentioning

confidence: 78%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 92%

Section: Runtime Out-of-distribution Detectionmentioning

confidence: 99%

See 2 more Smart Citations

Efficient Out-of-Distribution Detection Using Latent Space of β -VAE for Cyber-Physical Systems

Ramakrishna

Rahiminasab

Karsai

et al. 2022

ACM Trans. Cyber-Phys. Syst.

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, the dynamically changing operating conditions of the system at runtime potentially invalidate the design-time assumptions and the safety arguments [2]. A dynamic approach with proactive safety assessment components like Out-of-Distribution (OOD) detectors [3], [4], failure predictors [5], [6] and dynamic assurance monitors [7] is required for runtime safety assurance. Designing these components often requires labeled data from different operating modes of the system that belong to scenes with adverse operating conditions and sensor or actuator faults.…”

Section: Introductionmentioning

confidence: 99%

Risk-Aware Scene Sampling for Dynamic Assurance of Autonomous Systems

Ramakrishna¹,

Luo²,

Barve³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Autonomous Cyber-Physical Systems must often operate under uncertainties like sensor degradation and shifts in the operating conditions, which increases its operational risk. Dynamic Assurance of these systems requires designing runtime safety components like Out-of-Distribution detectors and risk estimators, which require labeled data from different operating modes of the system that belong to scenes with adverse operating conditions, sensors, and actuator faults. Collecting real-world data of these scenes can be expensive and sometimes not feasible. So, scenario description languages with samplers like random and grid search are available to generate synthetic data from simulators, replicating these real-world scenes. However, we point out three limitations in using these conventional samplers. First, they are passive samplers, which do not use the feedback of previous results in the sampling process. Second, the variables to be sampled may have constraints that are often not included. Third, they do not balance the tradeoff between exploration and exploitation, which we hypothesize is necessary for better search space coverage. We present a scene generation approach with two samplers called Random Neighborhood Search (RNS) and Guided Bayesian Optimization (GBO), which extend the conventional random search and Bayesian Optimization search to include the limitations. Also, to facilitate the samplers, we use a risk-based metric that evaluates how risky the scene was for the system. We demonstrate our approach using an Autonomous Vehicle example in CARLA simulation. To evaluate our samplers, we compared them against the baselines of random search, grid search, and Halton sequence search. Our samplers of RNS and GBO sampled a higher percentage of high-risk scenes of 83% and 92%, compared to 56% 66% and 71% of the grid, random and Halton samplers, respectively.

show abstract

SCARS: Suturing wounds due to conflicts between non‐functional requirements in autonomous and robotic systems

Roy,

Bag,

Deb

et al. 2023

Softw Pract Exp

View full text Add to dashboard Cite

In autonomous and robotic systems, the functional requirements (FRs) and non‐functional requirements (NFRs) are gathered from multiple stakeholders. The different stakeholder requirements are associated with different components of the robotic system and with the contexts in which the system may operate. This aggregation of requirements from different sources (multiple stakeholders) often results in inconsistent or conflicting sets of requirements. Conflicts among NFRs for robotic systems heavily depend on features of actual execution contexts. It is essential to analyze the inconsistencies and conflicts among the requirements in the early planning phase to design the robotic systems in a systematic manner. In this work, we design and experimentally evaluate a framework, called SCARS, providing: (a) a domain‐specific language extending the ROS2 Domain Specific Language (DSL) concepts by considering the different environmental contexts in which the system has to operate, (b) support to analyze their impact on NFRs, and (c) the computation of the optimal degree of NFR satisfaction that can be achieved within different system configurations. The effectiveness of SCARS has been validated on the iRobot Create3 robot using Gazebo simulation.

show abstract

ReSonAte: A Runtime Risk Assessment Framework for Autonomous Systems

Cited by 17 publications

References 33 publications

Efficient Out-of-Distribution Detection Using Latent Space of β -VAE for Cyber-Physical Systems

Efficient Out-of-Distribution Detection Using Latent Space of β -VAE for Cyber-Physical Systems

Risk-Aware Scene Sampling for Dynamic Assurance of Autonomous Systems

SCARS: Suturing wounds due to conflicts between non‐functional requirements in autonomous and robotic systems

Contact Info

Product

Resources

About