Defending Against Neural Network Model Stealing Attacks Using Deceptive Perturbations

Abstract

Machine learning architectures are readily available on the web, but creating the high quality training data is costly. However, a pretrained model on a cloud service can be used to generate labeled data to steal the model if the adversary can obtain output labels for chosen inputs. To protect against these attacks, it has been proposed to limit the information provided to the adversary by omitting probability scores, significantly impacting the utility of the provided service. In this work, we illustrate how a service provider can still provide useful, albeit misleading, class probability information, while significantly limiting the success of the attack. Our defense forces the adversary to discard the class probabilities, requiring significantly more queries before they can train a model with comparable performance. We evaluate several attack strategies, model architectures, and hyperparameters under varying adversarial models, and evaluate the efficacy of our defense against the strongest adversary. Finally, we quantify the amount of noise injected into the class probabilities to measure the loss in utility, e.g., adding 1.26 nats per query on CIFAR-10 and 3.27 on MNIST. Our evaluation shows our defense can degrade the accuracy of the stolen model at least 20%, or require up to 64 times more queries while keeping the accuracy of the protected model almost intact.

PRADA: Protecting Against DNN Model Stealing Attacks

“…A first defense to model extraction is to reduce the amount of information given to an adversary by modifying the model prediction. Prediction probabilities can be quantized [55] or perturbed to deceive to the adversary [29]. We have shown that model extraction attacks are effective even without using prediction probabilities (Sect.…”

Privacy and Security Issues in Deep Learning: A Survey

“…Tramer et al [15] and Lee et al [100] suggested that the efficiency of the model extraction attack can be decreased by omitting the confidence value or adding smart noise to the predicted probabilities. However, Juuti et al [101] shown that model extraction is effective, even omitting prediction probabilities.…”

Protecting Decision Boundary of Machine Learning Model With Differentially Private Perturbation

“…Confidence rounding and ensemble model were shown effective against equationsolving extractions in [1]. Lee et al [27] proposed perturbations using the mechanism of reverse sigmoid to inject deceptive noises to output confidence, which preserved the validity of top and bottom rank labels. Kesarwani et al [6] monitored user-server streams to evaluate the threat level of model extraction with two strategies based on entropy and compact model summaries.…”