Practical Blind Membership Inference Attack via Differential Comparisons

Abstract

Membership inference (MI) attacks affect user privacy by inferring whether given data samples have been used to train a target learning model, e.g., a deep neural network. There are two types of MI attacks in the literature, i.e., these with and without shadow models. The success of the former heavily depends on the quality of the shadow model, i.e., the transferability between the shadow and the target; the latter, given only blackbox probing access to the target model, cannot make an effective inference of unknowns, compared with MI attacks using shadow models, due to the insufficient number of qualified samples labeled with ground truth membership information. In this paper, we propose an MI attack, called BLINDMI, which probes the target model and extracts membership semantics via a novel approach, called differential comparison. The high-level idea is that BLINDMI first generates a dataset with nonmembers via transforming existing samples into new samples, and then differentially moves samples from a target dataset to the generated, non-member set in an iterative manner. If the differential move of a sample increases the set distance, BLINDMI considers the sample as non-member and vice versa. BLINDMI was evaluated by comparing it with state-of-theart MI attack algorithms. Our evaluation shows that BLINDMI improves F1-score by nearly 20% when compared to state-ofthe-art on some datasets, such as Purchase-50 and Birds-200, in the blind setting where the adversary does not know the target model's architecture and the target dataset's ground truth labels. We also show that BLINDMI can defeat state-of-the-art defenses.

Membership Inference Attacks and Defenses in Neural Network Pruning

“…In the evaluation, we consider the most widely used datasets, neural network architectures, and optimization approaches following recent research of MIAs [10,23,28,45].…”

Mitigating Membership Inference Attacks by Self-Distillation Through a Novel Ensemble Architecture

“…Jayaraman et al [20] analyze MIA in more realistic assumptions by relaxing proportion of training set size and testing set size in the MIA set up to be any positive value instead of 1. Hui et al [18] study MIA in a practical scenario, assuming no true labels of target samples are known and utilizing differential comparison for MIAs. Another threat model for MIAs is that of a white-box setting, i.e., attacker has full access to the model [26,33], which can exploit model parameters to infer membership information.…”

EncoderMI: Membership Inference against Pre-trained Encoders in Contrastive Learning

“…Membership inference: In membership inference against machine learning classifiers [11,18,19,22,24,31,32,32,35,35,41,42,44,47,48,52], an inferrer aims to infer whether an input is in the training dataset of a classifier (called target classifier). For instance, in the methods proposed by Shokri et al [44], an inferrer first trains shadow classifiers to mimic the behaviors of the target classifier.…”

“…Salem et al [42] further improved these methods by relaxing the assumptions about the inferrer. Hui et al [24] proposed blind membership inference methods that do not require training shadow classifiers. Concurrent to our work, He et al [22] also studied membership inference against contrastive learning.…”