Certifying the True Error: Machine Learning in Coq with Verified Generalization Guarantees

Bagnall, Alexander; Stewart, Gordon

doi:10.1609/aaai.v33i01.33012662

Cited by 21 publications

(22 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The partial prior knowledge is far from a complete prior distribution, thus it is easier to obtain from DNN lifecycle activities (C4). For instance, there are studies on the generalisation error bounds, based on how the DNN was constructed, trained and verified [22,5]. We present examples on how to obtain such partial prior knowledge (G6) using evidence, e.g.…”

Section: Cbi Utilising Operational Datamentioning

confidence: 99%

A Safety Framework for Critical Systems Utilising Deep Neural Networks

Zhao,

Banks,

Sharp

et al. 2020

Preprint

View full text Add to dashboard Cite

Increasingly sophisticated mathematical modelling processes from Machine Learning are being used to analyse complex data. However, the performance and explainability of these models within practical critical systems requires a rigorous and continuous verification of their safe utilisation. Working towards addressing this challenge, this paper presents a principled novel safety argument framework for critical systems that utilise deep neural networks. The approach allows various forms of predictions, e.g., future reliability of passing some demands, or confidence on a required reliability level. It is supported by a Bayesian analysis using operational data and the recent verification and validation techniques for deep learning. The prediction is conservative -it starts with partial prior knowledge obtained from lifecycle activities and then determines the worst-case prediction. Open challenges are also identified.

show abstract

Section: Cbi Utilising Operational Datamentioning

confidence: 99%

A Safety Framework for Critical Systems Utilising Deep Neural Networks

Zhao,

Banks,

Sharp

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Training networks using Keras made our work significantly easier, and importing the models to our libraries was an easy task. There is already existing work importing pre-trained models to theorem provers for the purposes of verification, e.g., MLCert in Coq [4]. Our approach to importing models differs from MLCert: we translate floating-point numbers to F * reals, whereas MLCert translates them to bit-vectors.…”

Section: Lessons Learnedmentioning

confidence: 99%

Neural Networks, Secure by Construction

Kokke

Komendantskaya

Kienitz

et al. 2020

Programming Languages and Systems

View full text Add to dashboard Cite

We present StarChild and Lazuli, two libraries which leverage refinement types to verify neural networks, implemented in F * and Liquid Haskell. Refinement types are types augmented, or refined, with assertions about values of that type, e.g., "integers greater than five", which are checked by an SMT solver. Crucially, these assertions are written in the language itself. A user of our library can refine the type of neural networks, e.g., "neural networks which are robust against adversarial attacks", and expect F * to handle the verification of this claim for any specific network, without having to change the representation of the network, or even having to learn about SMT solvers.Our initial experiments indicate that our approach could greatly reduce the burden of verifying neural networks. Unfortunately, they also show that SMT solvers do not scale to the sizes required for neural network verification.

show abstract

“…An alternative approach, that would resolve all three shortcomings identified in the previous section, is the ITP approach, as advocated in e.g. [1,2]. It amounts to first defining neural networks directly in Coq, and then proving properties about these definitions directly, thus avoiding programming in Python at the verification stage.…”

Section: Itp Approach To Neural Network Verificationmentioning

confidence: 99%

“…The answers include proofs of properties of generalisation bounds, equality of neural networks, properties of neural network architectures. Good examples are [1,2]. Usually, these verification projects are conducted in interactive theorem provers (ITPs), such as Coq [4], as they benefit from Coq's rich higher-order language and well-developed proof libraries.…”

Section: Introductionmentioning

confidence: 99%

Neural Network Verification for the Masses (of AI graduates)

Komendantskaya,

Stewart,

Duncan

et al. 2019

Preprint

View full text Add to dashboard Cite

Rapid development of AI applications has stimulated demand for, and has given rise to, the rapidly growing number and diversity of AI MSc degrees. AI and Robotics research communities, industries and students are becoming increasingly aware of the problems caused by unsafe or insecure AI applications. Among them, perhaps the most famous example is vulnerability of deep neural networks to "adversarial attacks". Owing to wide-spread use of neural networks in all areas of AI, this problem is seen as particularly acute and pervasive. Despite of the growing number of research papers about safety and security vulnerabilities of AI applications, there is a noticeable shortage of accessible tools, methods and teaching materials for incorporating verification into AI programs. LAIV -the Lab for AI and Verification -is a newly opened research lab at Heriot-Watt university that engages AI and Robotics MSc students in verification projects, as part of their MSc dissertation work. In this paper, we will report on successes and unexpected difficulties LAIV faces, many of which arise from limitations of existing programming languages used for verification. We will discuss future directions for incorporating verification into AI degrees.

show abstract

Certifying the True Error: Machine Learning in Coq with Verified Generalization Guarantees

Cited by 21 publications

References 13 publications

A Safety Framework for Critical Systems Utilising Deep Neural Networks

A Safety Framework for Critical Systems Utilising Deep Neural Networks

Neural Networks, Secure by Construction

Neural Network Verification for the Masses (of AI graduates)

Contact Info

Product

Resources

About