Hu-Fu: Hardware and Software Collaborative Attack Framework Against Neural Networks

Abstract

Recently, Deep Learning (DL), especially Convolutional Neural Network (CNN), develops rapidly and is applied to many tasks, such as image classification, face recognition, image segmentation, and human detection. Due to its superior performance, DL-based models have a wide range of application in many areas, some of which are extremely safety-critical, e.g. intelligent surveillance and autonomous driving. Due to the latency and privacy problem of cloud computing, embedded accelerators are popular in these safety-critical areas. However, the robustness of the embedded DL system might be harmed by inserting hardware/software Trojans into the accelerator and the neural network model, since the accelerator and deploy tool (or neural network model) are usually provided by third-party companies. Fortunately, inserting hardware Trojans can only achieve inflexible attack, which means that hardware Trojans can easily break down the whole system or exchange two outputs, but can't make CNN recognize unknown pictures as targets. Though inserting software Trojans has more freedom of attack, it often requires tampering input images, which is not easy for attackers. So, in this paper, we propose a hardware-software collaborative attack framework to inject hidden neural network Trojans, which works as a back-door without requiring manipulating input images and is flexible for different scenarios. We test our attack framework for image classification and face recognition tasks, and get attack success rate of 92.6% and 100% on CIFAR10 and YouTube Faces, respectively, while keeping almost the same accuracy as the unattacked model in the normal mode. In addition, we show a specific attack scenario in which a face recognition system is attacked and gives a specific wrong answer.

ABS: Scanning Neural Networks for Back-doors by Artificial Brain Stimulation

“…However this approach has only been tested on small synthetic model and not yet on real DNNs. Clean label attacks [51,58] [19,34] trojans hardware that neural networks are running on. It injects back-door by tampering with the circuits.…”

Ten years of hardware Trojans: a survey from the attacker's perspective

“…They exploit the statistical properties of each layer's output to trigger the HT, which makes the HT extremely stealthy. Li et al [117] proposed a more flexible attack framework on a neural network that combines the hardware and software. Particularly, in addition to the hardware HT circuit, Trojan weights are embedded in neural networks.…”

“…Particularly, in addition to the hardware HT circuit, Trojan weights are embedded in neural networks. The Trojan is only inserted in a part of the network and does not affect the overall accuracy, thus can ensure stealthy [117]. In the above attacks, the attacker needs to have the knowledge of the model.…”

FAdeML: Understanding the Impact of Pre-Processing Noise Filtering on Adversarial Machine Learning

“…One of the most common attacks is to exploit the data-dependency by manipulating/intruding the training dataset [16], [17], [18] or the corresponding labels [19]. Similarly, the baseline ML algorithm and training tools can also be attacked by adding new layers/nodes or manipulating the hyper-parameters [13], [20], [21], [22], as illustrated in Fig C. Motivating Pre-processing Noise Filter-Aware Adversarial ML Although most current adversarial ML security attacks incorporate pre-processing elements (such as shuffling, gray scaling, local histogram utilization and normalization) [23] in their design and assume that an attacker can access the output of the preprocessing noise filtering, getting this access requires hardware manipulations and is practically difficult. If an attacker, on the other hand, does not have hardware access to the pre-processing filters, it becomes very challenging to incorporate the effect of pre-processing along with noise filtering, which raises the following key research questions:…”