Verification for Machine Learning, Autonomy, and Neural Networks Survey

Xiang, Weiming; Musau, Patrick; Wild, Ayana A.; Lopez, Diego Manzanas; Hamilton, Nathaniel; Yang, Xiaodong; Rosenfeld, Joel; Johnson, Taylor T.

doi:10.48550/arxiv.1810.01989

Cited by 25 publications

(31 citation statements)

References 109 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While important for deployments with schemas, they do not apply to the settings we consider. Other work considers statistical measures of accuracy [1], fuzzing for numeric errors [18], worst case perturbations [31], data linting [10], and other techniques [19]. These approaches are complementary to Fixy.…”

Section: Testingmentioning

confidence: 99%

Finding Label and Model Errors in Perception Data With Learned Observation Assertions

Kang¹,

Aréchiga²,

Pillai³

et al. 2022

Preprint

View full text Add to dashboard Cite

ML is being deployed in complex, real-world scenarios where errors have impactful consequences. In these systems, thorough testing of the ML pipelines is critical. A key component in ML deployment pipelines is the curation of labeled training data. Common practice in the ML literature assumes that labels are the ground truth. However, in our experience in a large autonomous vehicle development center, we have found that vendors can often provide erroneous labels, which can lead to downstream safety risks in trained models.To address these issues, we propose a new abstraction, learned observation assertions, and implement it in a system called Fixy. Fixy leverages existing organizational resources, such as existing (possibly noisy) labeled datasets or previously trained ML models, to learn a probabilistic model for finding errors in human-or modelgenerated labels. Given user-provided features and these existing resources, Fixy learns feature distributions that specify likely and unlikely values (e.g., that a speed of 30mph is likely but 300mph is unlikely). It then uses these feature distributions to score labels for potential errors. We show that Fixy can automatically rank potential errors in real datasets with up to 2× higher precision compared to recent work on model assertions and standard techniques such as uncertainty sampling.

show abstract

Section: Testingmentioning

confidence: 99%

Finding Label and Model Errors in Perception Data With Learned Observation Assertions

Kang¹,

Aréchiga²,

Pillai³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…[76] provides an excellent software repository for general computation graphs. When NNs are embedded in feedback loops, a new set of challenges arise for verification, surveyed in [77], [78]. To verify stability properties, [79] uses linear differential inclusions (LDI), and [80] uses interval constraint programming (ICP).…”

Section: Verification Of Neural Feedback Loopsmentioning

confidence: 99%

Neural Network Verification in Control

Everett¹

2021

Preprint

View full text Add to dashboard Cite

Learning-based methods could provide solutions to many of the long-standing challenges in control. However, the neural networks (NNs) commonly used in modern learning approaches present substantial challenges for analyzing the resulting control systems' safety properties. Fortunately, a new body of literature could provide tractable methods for analysis and verification of these high dimensional, highly nonlinear representations. This tutorial first introduces and unifies recent techniques (many of which originated in the computer vision and machine learning communities) for verifying robustness properties of NNs. The techniques are then extended to provide formal guarantees of neural feedback loops (e.g., closed-loop system with NN control policy). The provided tools are shown to enable closed-loop reachability analysis and robust deep reinforcement learning.

show abstract

“…Academic research on verification & validation tailored for AI systems has received a head start compared to requirements engineering for AI. New papers continuously appear, and secondary studies on AI testing [43,44] and AI verification [45] reveal hundreds of publications. As automation is close at hand for verification & validation solutions, the primary purpose of the pipelines in the metaphor is to stress that they shall reach all the way to the requirements engineering buttress.…”

Section: Reinforced Ai Systems Using Buttresses and Rebarsmentioning

confidence: 99%

Agility in Software 2.0 -- Notebook Interfaces and MLOps with Buttresses and Rebars

Borg¹

2021

Preprint

View full text Add to dashboard Cite

Artificial intelligence through machine learning is increasingly used in the digital society. Solutions based on machine learning bring both great opportunities, thus coined "Software 2.0," but also great challenges for the engineering community to tackle. Due to the experimental approach used by data scientists when developing machine learning models, agility is an essential characteristic. In this keynote address, we discuss two contemporary development phenomena that are fundamental in machine learning development, i.e., notebook interfaces and MLOps. First, we present a solution that can remedy some of the intrinsic weaknesses of working in notebooks by supporting easy transitions to integrated development environments. Second, we propose reinforced engineering of AI systems by introducing metaphorical buttresses and rebars in the MLOps context. Machine learning-based solutions are dynamic in nature, and we argue that reinforced continuous engineering is required to quality assure the trustworthy AI systems of tomorrow.1 bit.ly/3dKeUEH

show abstract

Verification for Machine Learning, Autonomy, and Neural Networks Survey

Cited by 25 publications

References 109 publications

Finding Label and Model Errors in Perception Data With Learned Observation Assertions

Finding Label and Model Errors in Perception Data With Learned Observation Assertions

Neural Network Verification in Control

Agility in Software 2.0 -- Notebook Interfaces and MLOps with Buttresses and Rebars

Contact Info

Product

Resources

About