Finding the most likely path to a set of failure states is important to the analysis of safety-critical systems that operate over a sequence of time steps, such as aircraft collision avoidance systems and autonomous cars. In many applications such as autonomous driving, failures cannot be completely eliminated due to the complex stochastic environment in which the system operates. As a result, safety validation is not only concerned about whether a failure can occur, but also discovering which failures are most likely to occur. This article presents adaptive stress testing (AST), a framework for finding the most likely path to a failure event in simulation. We consider a general black box setting for partially observable and continuous-valued systems operating in an environment with stochastic disturbances. We formulate the problem as a Markov decision process and use reinforcement learning to optimize it. The approach is simulation-based and does not require internal knowledge of the system, making it suitable for black-box testing of large systems. We present different formulations depending on whether the state is fully observable or partially observable. In the latter case, we present a modified Monte Carlo tree search algorithm that only requires access to the pseudorandom number generator of the simulator to overcome partial observability. We also present an extension of the framework, called differential adaptive stress testing (DAST), that can find failures that occur in one system but not in another. This type of differential analysis is useful in applications such as regression testing, where we are concerned with finding areas of relative weakness compared to a baseline. We demonstrate the effectiveness of the approach on an aircraft collision avoidance application, where a prototype aircraft collision avoidance system is stress tested to find the most likely scenarios of near mid-air collision.
Finding the most likely path to a set of failure states is important to the analysis of safety-critical systems that operate over a sequence of time steps, such as aircraft collision avoidance systems and autonomous cars. While efficient solutions exist for certain classes of systems, a scalable general solution for stochastic, partially observable, and continuous-valued systems remains challenging. Existing formal and simulation-based methods either cannot scale to large systems or are computationally inefficient. This paper presents adaptive stress testing (AST), a framework for searching a simulator for the most likely path to a failure event. We formulate the problem as a Markov decision process and use reinforcement learning to optimize it. The approach is simulation-based and does not require internal knowledge of the system, making it suitable for black box testing of large systems. We present formulations for both systems where the state is fully observable and partially observable. In the latter case, we present a modified Monte Carlo tree search algorithm, called Monte Carlo tree search for seed-action simulators (MCTS-SA), that only requires access to the pseudorandom number generator of the simulator to overcome partial observability. We also present an extension of the framework, called differential adaptive stress testing (DAST), that can be used to find failures that occur in one system but not in another. This type of differential analysis is useful in applications such as regression testing, where we are concerned with finding areas of relative weakness compared to a baseline. We demonstrate the effectiveness of the approach on an aircraft collision avoidance application, where a prototype aircraft collision avoidance system is stress tested to find the most likely failure scenarios.
The next-generation Airborne Collision Avoidance System (ACAS X) is currently being developed and tested to replace the Traffic Alert and Collision Avoidance System (TCAS) as the next international standard for collision avoidance. To validate the safety of the system, stress testing in simulation is one of several approaches for analyzing near mid-air collisions (NMACs). Understanding how NMACs can occur is important for characterizing risk and informing development of the system. Recently, adaptive stress testing (AST) has been proposed as a way to find the most likely path to a failure event. The simulation-based approach accelerates search by formulating stress testing as a sequential decision process then optimizing it using reinforcement learning. The approach has been successfully applied to stress test a prototype of ACAS X in various simulated aircraft encounters. In some applications, we are not as interested in the system's absolute performance as its performance relative to another system. Such situations arise, for example, during regression testing or when deciding whether a new system should replace an existing system. In our collision avoidance application, we are interested in finding cases where ACAS X fails but TCAS succeeds in resolving a conflict. Existing approaches do not provide an efficient means to perform this type of analysis. This paper extends the AST approach to differential analysis by searching two simulators simultaneously and maximizing the difference between their outcomes. We call this approach differential adaptive stress testing (DAST). We apply DAST to compare a prototype of ACAS X against TCAS and show examples of encounters found by the algorithm.
Abstract-In this paper we study the problem of dynamic optimization of ping schedule in an active sonar buoy network deployed to provide persistent surveillance of a littoral area through multistatic detection. The goal of ping scheduling is to dynamically determine when to ping and which ping source to engage in order to achieve the desirable detection performance. For applications where persistent surveillance is needed for an extended period of time, it is expected that the energy available at each ping source is limited relative to the required system lifetime. Hence efficient management of power consumption for pinging is important to support the required lifetime of the network while maintaining acceptable detection performance. Our approach to ping optimization is based on the application of approximate Partially Observable Markov Decision Process (POMDP) techniques such as the rollout algorithm. To enable a practical implementation of the policy rollout, we apply sampling-based techniques based on a simplified model that approximates the detailed multistatic model. Using high fidelity sonar simulations, we evaluate the performance of the proposed approach and compare it with the greedy technique in terms of detection performance and system lifetime.
The activity of mitochondrial complex I of the electron transport chain (ETC) is known to be affected by an extraordinarily large number of diverse xenobiotics, and dysfunction at complex I has been associated with a variety of disparate human diseases, including those with potentially environmentally relevant etiologies. However, the risks associated with mixtures of complex I inhibitors have not been fully explored, and this warrants further examination of potentially greater than additive effects that could lead to toxicity. A potential complication for the prediction of mixture effects arises because mammalian mitochondrial complex I has been shown to exist in two distinct dynamic conformations based upon substrate availability. In this study, we tested the accepted models of additivity as applied to mixtures of rotenone, deguelin, and pyridaben, with and without substrate limitation. These compounds represent both natural and synthetic inhibitors of complex I of the ETC, and experimental evidence to date indicates that these inhibitors share a common binding domain with partially overlapping binding sites. Therefore, we hypothesized that prediction of their mixtures effects would follow dose addition. Using human hepatocytes, we analyzed the effects of these mixtures at doses between 0.001 and 100 μM on overall cellular viability. Analysis of the dose-response curves resulting from challenge with all possible binary and ternary mixtures revealed that the appropriate model was not clear. All of the mixtures tested were found to be in agreement with response addition, but only rotenone plus deguelin and the ternary mixture followed dose addition. To determine if conformational regulation via substrate limitation could improve model selection and our predictions, we tested the models of additivity for the binary and ternary mixtures of inhibitors when coexposed with 2-deoxy-d-glucose (2-DG), which limits NADH via upstream inhibition of glycolysis. Coexposure of inhibitors with 2-DG did facilitate model selection: Rotenone plus pyridaben and the ternary mixture were in sole agreement with dose addition, while deguelin plus pyridaben was in sole agreement with response addition. The only ambiguous result was the agreement of both models with the mixture of rotenone plus deguelin with 2-DG, which may be explained by deguelin's well-known affinity for protein kinase B (Akt) in addition to complex I. Thus, our findings indicate that predictive models for mixtures of mitochondrial complex I inhibitors appear to be compound specific, and our research highlights the need to control for dynamic conformational changes to improve our mechanistic understanding of additivity with these inhibitors.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.