FANNet: Formal Analysis of Noise Tolerance, Training Bias and Input Sensitivity in Neural Networks

Naseer, Mahum; Minhas, Mishal Fatima; Khalid, Faiq; Hanif, Muhammad Abdullah; Hasan, Osman; Shafique, Muhammad

doi:10.23919/date48585.2020.9116247

Cited by 15 publications

(8 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They presented theoretical results with respect to the approximations of ReLU NNs and implemented a solver to verify ReLU NNs. Naseer et al [20] proposed a methodology that leverages model checking to perform formal NN analyses under diferent input noise ranges to rigorously analyze the noise tolerance of NNs. Yin et al [21] utilized a probabilistic model-checking tool PRISM to verify the reliability of an artiicial neural network (ANN)-based system in medical diagnosis.…”

Section: Related Workmentioning

confidence: 99%

A Novel GAPG Approach to Automatic Property Generation for Formal Verification: The GAN Perspective

Gao

Dai

Miao

et al. 2023

ACM Trans. Multimedia Comput. Commun. Appl.

View full text Add to dashboard Cite

Formal methods have been widely used to support software testing to guarantee correctness and reliability. For example, model checking technology attempts to ensure that the verification property of a specific formal model is satisfactory for discovering bugs or abnormal behavior from the perspective of temporal logic. However, because automatic approaches are lacking, a software developer/tester must manually specify verification properties. A generative adversarial network (GAN) learns features from input training data and outputs new data with similar or coincident features. GANs have been successfully used in the image processing and text processing fields and achieved interesting and automatic results. Inspired by the power of GANs, in this paper, we propose a GAN-based automatic property generation (GAPG) approach to generate verification properties supporting model checking. First, the verification properties in the form of computational tree logic (CTL) are encoded and used as input to the GAN. Second, we introduce regular expressions as grammar rules to check the correctness of the generated properties. These rules work to detect and filter meaningless properties that occur because the GAN learning process is uncontrollable and may generate unsuitable properties for real applications. Third, the learning network is further trained by using labeled information associated with the input properties. These are intended to guide the training process to generate additional new properties, particularly those that map to corresponding formal models. Finally, a series of comprehensive experiments demonstrate that the proposed GAPG method can obtain new verification properties from two aspects: 1) using only CTL formulas and 2) using CTL formulas combined with Kripke structures.

show abstract

Section: Related Workmentioning

confidence: 99%

A Novel GAPG Approach to Automatic Property Generation for Formal Verification: The GAN Perspective

Gao

Dai

Miao

et al. 2023

ACM Trans. Multimedia Comput. Commun. Appl.

View full text Add to dashboard Cite

show abstract

“…This section describes the notions and provides the relevant formalism for balanced datasets, robustness, robustness bias, bias estimation and noise tolerance (Nanda et al, 2021;Naseer et al, 2020), which form the basis of Unbiased-Nets. The terminology and notations introduced in the section will be used throughout the rest of the paper.…”

Section: Preliminariesmentioning

confidence: 99%

“…These misclassifying noise patterns (i.e., the counterexamples) act as inputs for the counterexample analysis. These noise patterns can be collected either using a formal framework (such as the ones based on model checking used by Naseer et al (2020) and Bhatti, Naseer, Shafique, and Hasan (2022)) or an empirical approach (like the Fast Gradient Sign Method (FGSM) attack (Goodfellow, Shlens, & Szegedy, 2015)).…”

Section: Bias Detectionmentioning

confidence: 99%

UnbiasedNets: a dataset diversification framework for robustness bias alleviation in neural networks

et al. 2023

Self Cite

View full text Add to dashboard Cite

Performance of trained neural network (NN) models, in terms of testing accuracy, has improved remarkably over the past several years, especially with the advent of deep learning. However, even the most accurate NNs can be biased toward a specific output classification due to the inherent bias in the available training datasets, which may propagate to the real-world implementations. This paper deals with the robustness bias, i.e., the bias exhibited by the trained NN by having a significantly large robustness to noise for a certain output class, as compared to the remaining output classes. The bias is shown to result from imbalanced datasets, i.e., the datasets where all output classes are not equally represented. Towards this, we propose the UnbiasedNets framework, which leverages K-means clustering and the NN’s noise tolerance to diversify the given training dataset, even from relatively smaller datasets. This generates balanced datasets and reduces the bias within the datasets themselves. To the best of our knowledge, this is the first framework catering to the robustness bias problem in NNs. We use real-world datasets to demonstrate the efficacy of the UnbiasedNets for data diversification, in case of both binary and multi-label classifiers. The results are compared to well-known tools aimed at generating balanced datasets, and illustrate how existing works have limited success while addressing the robustness bias. In contrast, UnbiasedNets provides a notable improvement over existing works, while even reducing the robustness bias significantly in some cases, as observed by comparing the NNs trained on the diversified and original datasets.

show abstract

“…The approach gives promising results but suffers from its complexity, and the verification is exponential in the number of features. [159] propose FANNET, a formal analysis framework that uses model checking to evaluate the noise tolerance of a trained neural network. During the verification process, in case of non-satisfiability of the input property, a counterexample is generated.…”

Section: Smt-based Model Checking: Direct Verificationmentioning

confidence: 99%

How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review

Tambon,

Laberge,

et al. 2021

Preprint

View full text Add to dashboard Cite

Context: Machine Learning (ML) has been at the heart of many innovations over the past years. However, including it in so-called "safety-critical" systems such as automotive or aeronautic has proven to be very challenging, since the shift in paradigm that ML brings completely changes traditional certification approaches.Objective: This paper aims to elucidate challenges related to the certification of ML-based safety-critical systems, as well as the solutions that are proposed in the literature to tackle them, answering the question "How to Certify Machine Learning Based Safety-critical Systems?".Method: We conduct a Systematic Literature Review (SLR) of research papers published between 2015 to 2020, covering topics related to the certification of ML systems. In total, we identified 217 papers covering topics considered to be the main pillars of ML certification: Robustness, Uncertainty, Explainability, Verification, Safe Reinforcement Learning, and Direct Certification. We analyzed the main trends and problems of each sub-field and provided summaries of the papers extracted.Results: The SLR results highlighted the enthusiasm of the community for this subject, as well as the lack of diversity in term of datasets and type of ML models. It also emphasized the need to further develop connections between academia and industries to deepen the domain study. Finally, it also illustrated This work is supported by the DEEL project CRDPJ 537462-18 funded by the National Science and Engineering Research Council of Canada (NSERC) and the Consortium for Research and Innovation in Aerospace in Québec (CRIAQ), together with its industrial partners Thales Canada inc, Bell Textron Canada Limited, CAE inc and Bombardier inc.

show abstract

FANNet: Formal Analysis of Noise Tolerance, Training Bias and Input Sensitivity in Neural Networks

Cited by 15 publications

References 19 publications

A Novel GAPG Approach to Automatic Property Generation for Formal Verification: The GAN Perspective

A Novel GAPG Approach to Automatic Property Generation for Formal Verification: The GAN Perspective

UnbiasedNets: a dataset diversification framework for robustness bias alleviation in neural networks

How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review

Contact Info

Product

Resources

About