models of system-level behaviour have applications in design exploration, analysis, testing and verification. We describe a new algorithm for automatically extracting useful models, as automata, from execution traces of a HW/SW system driven by software exercising a usecase of interest. Our algorithm leverages modern program synthesis techniques to generate predicates on automaton edges, succinctly describing system behaviour. It employs trace segmentation to tackle complexity for long traces. We learn concise models capturing transaction-level, systemwide behaviourexperimentally demonstrating the approach using traces from a variety of sources, including the x86 QEMU virtual platform and the Real-Time Linux kernel.

A complex system is characterized by emergence of global properties which are very difficult, if not impossible, to anticipate just from complete knowledge of component behaviors. Emergence, hierarchical organization and numerosity are some of the characteristics of complex systems. Recently, there has been an exponential increase on the adoption of various neural network‐based machine learning models to govern the functionality and behavior of systems. With this increasing system complexity, achieving confidence in systems becomes even more difficult. Further, ease of interconnectivity among systems is permeating numerous system‐of‐systems, wherein multiple independent systems are expected to interact and collaborate to achieve unparalleled levels of functionality. Traditional verification and validation approaches are often inadequate to bring in the nuances of potential emergent behavior in a system‐of‐system, which may be positive or negative. This paper describes a novel approach towards application of machine learning based classifiers and formal methods for analyzing and evaluating emergent behavior of complex system‐of‐systems that comprise a hybrid of constituent systems governed by conventional models and machine learning models. The proposed approach involves developing a machine learning classifier model that learns on potential negative and positive emergent behaviors, and predicts the behavior exhibited. A formal verification model is then developed to assert negative emergent behavior. The approach is illustrated through the case of a swarm of autonomous UAVs flying in a formation, and dynamically changing the shape of the formation, to support varying mission scenarios. The effectiveness and performance of the approach are quantified.

No abstract

A complex system is characterized by emergence of global properties which are very difficult, if not impossible, to anticipate just from complete knowledge of component behaviors. Emergence, hierarchical organization and numerosity are some of the characteristics of complex systems. With increasing system complexity, achieving confidence in systems becomes increasingly difficult. With the recent trend towards significant footprint of complex system's functionality being governed by machine learning based models and algorithms, there is a need to ensure that emergent behavior associated with such systems are well analyzed and understood. Traditional verification and validation approaches are often inadequate to bring in the nuances of potential emergent behavior, which may be positive or negative. This paper describes a novel approach towards application of formal methods for analyzing and evaluating emergent behavior of complex systems that are governed by machine learning models. The proposed approach involves developing a machine learning classifier model that learns on potential negative and positive emergent behaviors, and leveraging the classifier in a formal verification model checking environment to assert negative emergent behavior. The approach is illustrated through an example of a pitch control system of an aircraft. The effectiveness and performance of the approach are quantified.

No abstract