BadNets: Evaluating Backdooring Attacks on Deep Neural Networks

Gu, Tianyu; Liu, Kang; Dolan-Gavitt, Brendan; Garg, Siddharth

doi:10.1109/access.2019.2909068

Cited by 718 publications

(783 citation statements)

References 27 publications

Supporting

Mentioning

659

Contrasting

Order By: Relevance

“…Existing Backdoor Attacks. Gu et al proposed BadNets that injects a backdoor to a DNN model by poisoning its training dataset [19]. The attacker first chooses a target label and a trigger pattern (i.e.…”

Section: Backdoor Attacks On Dnnmentioning

confidence: 99%

“…Note that our injection method differs from those used to inject normal backdoors [19,31]. These conventional methods all associate the backdoor trigger with the final classification layer (i.e.…”

Section: Attack Workflowmentioning

confidence: 99%

“…Digit. This application is commonly used in studying DNN vulnerabilities including normal backdoors [19,50]. Both Teacher and Student tasks are to recognize hand-written digits, where Teacher Table 1: Summary of tasks, models, and datasets used in our evaluation using four tasks.…”

Section: Experiments Setupmentioning

confidence: 99%

“…Finally, we use the Teacher training images as the non-target data X \y t . The Teacher model is a standard 4-layer CNN ( Table 6 in Appendix), used by previous works to evaluate conventional backdoor attacks [19]. The released Teacher model also instructs that transfer learning should freeze the first three layers and only fine-tune the last layer.…”

Section: Experiments Setupmentioning

confidence: 99%

“…launch a conventional backdoor attack against this application (e.g., via BadNets [19]), the attacker needs to have access to the selfdriving car's model training data and/or control its model training.…”

Section: Traffic Sign Recognitionmentioning

confidence: 99%

See 4 more Smart Citations

Latent Backdoor Attacks on Deep Neural Networks

Yao

Zheng

et al. 2019

Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security

267

169

View full text Add to dashboard Cite

Recent work has proposed the concept of backdoor attacks on deep neural networks (DNNs), where misbehaviors are hidden inside "normal" models, only to be triggered by very specific inputs. In practice, however, these attacks are difficult to perform and highly constrained by sharing of models through transfer learning. Adversaries have a small window during which they must compromise the student model before it is deployed.In this paper, we describe a significantly more powerful variant of the backdoor attack, latent backdoors, where hidden rules can be embedded in a single "Teacher" model, and automatically inherited by all "Student" models through the transfer learning process. We show that latent backdoors can be quite effective in a variety of application contexts, and validate its practicality through real-world attacks against traffic sign recognition, iris identification of lab volunteers, and facial recognition of public figures (politicians). Finally, we evaluate 4 potential defenses, and find that only one is effective in disrupting latent backdoors, but might incur a cost in classification accuracy as tradeoff.

show abstract

Section: Backdoor Attacks On Dnnmentioning

confidence: 99%

Section: Attack Workflowmentioning

confidence: 99%

Section: Experiments Setupmentioning

confidence: 99%

Section: Experiments Setupmentioning

confidence: 99%

Section: Traffic Sign Recognitionmentioning

confidence: 99%

See 3 more Smart Citations

Latent Backdoor Attacks on Deep Neural Networks

Yao

Zheng

et al. 2019

Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security

267

169

View full text Add to dashboard Cite

show abstract

A framework for fostering transparency in shared artificial intelligence models by increasing visibility of contributions

Barclay

Taylor

Preece

et al. 2020

Concurrency and Computation

View full text Add to dashboard Cite

Increased adoption of artificial intelligence (AI) systems into scientific workflows will result in an increasing technical debt as the distance between the data scientists and engineers who develop AI system components and scientists, researchers and other users grows. This could quickly become problematic, particularly where guidance or regulations change and once-acceptable best practice becomes outdated, or where data sources are later discredited as biased or inaccurate. This paper presents a novel method for deriving a quantifiable metric capable of ranking the overall transparency of the process pipelines used to generate AI systems, such that users, auditors and other stakeholders can gain confidence that they will be able to validate and trust the data sources and contributors in the AI systems that they rely on. The methodology for calculating the metric, and the type of criteria that could be used to make judgements on the visibility of contributions to systems are evaluated through models published at ModelHub and PyTorch Hub, popular archives for sharing science resources, and is found to be helpful in driving consideration of the contributions made to generating AI systems and approaches toward effective documentation and improving transparency in machine learning assets shared within scientific communities. K E Y W O R D Saccountability, data ecosystems, data provenance, ML model evaluation, model zoo, transparency INTRODUCTIONScientists and researchers across all fields are making use of artificial intelligence (AI) systems and machine learning (ML) models in their experimentation. Increasingly, these new research assets are created by domain experts curating and aggregating data from multiple and diverse sources; their outputs can be new data assets or heavily data-influenced products, including ML models, which go on to be used in research within the originating organization, or in the wider community when they are published or distributed and shared via gateways with collaborating or even unknown third party research groups and organizations. This is already a real situation-in the UK financial services sector, for example, 24% of ML use cases are developed and implemented by third-party providers, with many of those developing internally also reporting adaptation or further development of off-the-shelf ML models or libraries. 1Providing support to offer transparency and traceability of assets through the production pipeline is an important contributor to delivering accountability, which is necessary to achieve and retain confidence and trust, such that scientists and practitioners using AI systems, whether developed in-house or sourced externally from their community, are able to demonstrate the provenance and authenticity of the data and knowledge they use to make decisions. 2,3 In considering the development workflows for ML models, the contributing assets can be identified and itemized, and will typically include data sources and labeled datasets used for model training and validation, alon...

show abstract

Securing AI‐based healthcare systems using blockchain technology: A state‐of‐the‐art systematic literature review and future research directions

Shinde,

Patil,

Kotecha

et al. 2023

Trans Emerging Tel Tech

View full text Add to dashboard Cite

Healthcare institutions are progressively integrating artificial intelligence (AI) into their operations. The extraordinary potential of AI is restricted by insufficient medical data for AI model training and adversarial attacks wherein attackers perturb the dataset by adding some noise to it, which leads to the malfunctioning of the AI models, and a lack of trust caused by the opaque operational approach it employs. This Systematic Literature Review (SLR) is a state‐of‐the‐art survey of the research on blockchain technology for securing AI‐integrated healthcare applications. The most relevant articles from the Scopus and Web of Science (WoS) databases were identified using the PRISMA model. Most of the existing literature is about protecting the healthcare data used by AI‐based healthcare systems using blockchain technology, but the modality of data (text, images, audio, and sound) was not specifically mentioned. Information on protecting the training phase and model deployment for AI‐based healthcare systems considering the variations in feature extraction based on the modality of data was also not clearly specified. Hence, the three subfields of AI, namely, natural language processing (NLP), computer vision, and acoustic AI are further studied to identify security loopholes in its implementation pipeline. The three phases, namely the dataset, the training phase, and the trained models need to be protected from adversaries to avoid malfunctioning of the deployed AI models. The nature of the data processed by NLP, computer vision, and acoustic AI, underlying deep neural network (DNN) architectures, the complexity of attacks, and the perceivability of attacks by humans are analyzed to identify the need for security. A blockchain solution for AI‐based healthcare systems is synthesized based on the findings that have demonstrated the distinctive technological features of blockchains. It offers a solution for the privacy and security issues encountered by NLP, computer vision, and acoustic AI to boost the widespread adoption of AI applications in healthcare.

show abstract

BadNets: Evaluating Backdooring Attacks on Deep Neural Networks

Cited by 718 publications

References 27 publications

Latent Backdoor Attacks on Deep Neural Networks

Latent Backdoor Attacks on Deep Neural Networks

A framework for fostering transparency in shared artificial intelligence models by increasing visibility of contributions

Securing AI‐based healthcare systems using blockchain technology: A state‐of‐the‐art systematic literature review and future research directions

Contact Info

Product

Resources

About