Deep Validation: Toward Detecting Real-World Corner Cases for Deep Neural Networks

Wu, Wei; Xu, Hui; Zhong, Sanqiang; Lyu, Michael R.; King, Irwin

doi:10.1109/dsn.2019.00026

Cited by 39 publications

(20 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most relevant, K. Pei et al [35] apply input space reduction techniques to transform the image, and can simulate a wide range of real-world distortions, noises, and deformations. W. Wu et al [36] present a faults model for deep neural networks classifiers, which includes several corner cases based on the alteration of the input image, including amongst the possible causes brightness, camera alignment, and object movements. Finally, evasion attacks consist in modifying the input to a classifier such that it is misclassified, while keeping the modification as small as possible [37].…”

Section: Related Workmentioning

confidence: 99%

“…Several works explore how to make the trained agents robust to artificially crafted or accidentally manipulated input images [35], [36], [37], and how to secure a camera from direct attacks that may disrupt the proper behavior of the camera itself [18], [19]. In fact, even slight alterations of the images may alter the output of the trained agent [39].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

RGB Cameras Failures and Their Effects in Autonomous Driving Applications

Secci

Ceccarelli

2023

IEEE Trans. Dependable and Secure Comput.

View full text Add to dashboard Cite

RGB cameras are one of the most relevant sensors for autonomous driving applications. It is undeniable that failures of vehicle cameras may compromise the autonomous driving task, possibly leading to unsafe behaviors when images that are subsequently processed by the driving system are altered. To support the definition of safe and robust vehicle architectures and intelligent systems, in this paper we define the failure modes of a vehicle camera, together with an analysis of effects and known mitigations. Further, we build a software library for the generation of the corresponding failed images, and we feed them to six object detectors for mono and stereo cameras and to the self-driving agent of an autonomous driving simulator. The resulting misbehaviors with respect to operating with clean images allow a better understanding of failures effects and the related safety risks in image-based applications.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

RGB Cameras Failures and Their Effects in Autonomous Driving Applications

Secci

Ceccarelli

2023

IEEE Trans. Dependable and Secure Comput.

View full text Add to dashboard Cite

show abstract

“…The type of test cases that these studies addressed were adversarial test cases in 14 occasions (e.g. [48,53,70]) and corner test cases in 3 occasions [22,213,219]. As opposed to the related categories about test case generation and selection, papers in this general category were very focused on conceptualizing different types of test cases.…”

Section: Software Testing (115 Studies)mentioning

confidence: 99%

Software Engineering for AI-Based Systems: A Survey

Martínez-Fernández,

Bogner,

Franch

et al. 2021

Preprint

View full text Add to dashboard Cite

AI-based systems are software systems with functionalities enabled by at least one AI component (e.g., for image-and speechrecognition, and autonomous driving). AI-based systems are becoming pervasive in society due to advances in AI. However, there is limited synthesized knowledge on Software Engineering (SE) approaches for building, operating, and maintaining AI-based systems.To collect and analyze state-of-the-art knowledge about SE for AI-based systems, we conducted a systematic mapping study. We considered 248 studies published between January 2010 and March 2020. SE for AI-based systems is an emerging research area, where more than 2/3 of the studies have been published since 2018. The most studied properties of AI-based systems are dependability and safety. We identified multiple SE approaches for AI-based systems, which we classified according to the SWEBOK areas. Studies related to software testing and software quality are very prevalent, while areas like software maintenance seem neglected. Data-related issues are the most recurrent challenges. Our results are valuable for: researchers, to quickly understand the state of the art and learn which topics need more research; practitioners, to learn about the approaches and challenges that SE entails for AI-based systems; and, educators, to bridge the gap among SE and AI in their curricula. CCS Concepts: • Software and its engineering → Software creation and management; • Computing methodologies → Machine learning;

show abstract

“…The discrepancy between the assumed input data in the development phase and the real input data observed in the operation phase likely causes undesirable outputs at runtime. Wu et al leveraged the data validation technique to detect real-world corner cases for DNN-based systems [24]. Considering a cyber-physical system employs ML components, Dreossi et al proposed a method to analyze the input space for ML classifiers that can lead to undesirable consequences in the cyber-physical system [25].…”

Section: Related Workmentioning

confidence: 99%

Using Diversities to Model the Reliability of N-version Machine Learning System

Machida¹

2021

Preprint

View full text Add to dashboard Cite

N-version machine learning system (MLS) is an architectural approach to reduce error outputs from a system by redundant configuration using multiple machine learning (ML) modules. Improved system reliability achieved by N-version MLS inherently depends on how diverse ML models are employed and how diverse input data sets are given. However, neither error input spaces of individual ML models nor input data distributions are obtainable in practice, which is a fundamental barrier to understanding the reliability gain by N-version architecture. In this paper, we introduce two diversity measures quantifying the similarities of ML models’ capabilities and the interdependence of input data sets, respectively. The defined measures are used to formulate the reliability of an elemental N-version MLS called dependent double-modules double-inputs MLS. The system is assumed to fail when two ML modules output errors simultaneously for the same classification task. The reliabilities of different architecture options for this MLS are comprehensively analyzed through a compact matrix representation form of the proposed reliability model. Except for limiting cases, we observe that the architecture exploiting two diversities tends to achieve preferable reliability under reasonable assumptions. Intuitive relations between diversity parameters and architecture reliabilities are also demonstrated through numerical experiments with hypothetical settings.

show abstract

Deep Validation: Toward Detecting Real-World Corner Cases for Deep Neural Networks

Cited by 39 publications

References 34 publications

RGB Cameras Failures and Their Effects in Autonomous Driving Applications

RGB Cameras Failures and Their Effects in Autonomous Driving Applications

Software Engineering for AI-Based Systems: A Survey

Using Diversities to Model the Reliability of N-version Machine Learning System

Contact Info

Product

Resources

About