Knowledge Distillation with BERT for Image Tag-Based Privacy Prediction

Abstract

Text in the form of tags associated with online images is often informative for predicting private or sensitive content from images. When using privacy prediction systems running on social networking sites that decide whether each uploaded image should get posted or be protected, users may be reluctant to share real images that may reveal their identity, but may share image tags. In such cases, privacy-aware tags become good indicators of image privacy and can be utilized to generate privacy decisions. In this paper, our aim is to learn tag representations for images to improve tagbased image privacy prediction. To achieve this, we explore self-distillation with BERT, in which we utilize knowledge in the form of soft probability distributions (soft labels) from the teacher model to help with the training of the student model. Our approach effectively learns better tag representations with improved performance on private image identification and outperforms state-of-the-art models for this task. Moreover, we utilize the idea of knowledge distillation to improve tag representations in a semi-supervised learning task. Our semi-supervised approach with only 20% of annotated data achieves similar performance compared with its supervised learning counterpart. Last, we provide a comprehensive analysis to get a better understanding of our approach.

Can Model Compression Improve NLP Fairness

“…Following works continue to use Knowledge Distillation to improve generalization (Arani et al, 2019) and robustness (Goldblum et al (2020)). Knowledge Distillation is also used to improve models on privacy protection (Shejwalkar and Houmansadr (2019), Zhao and Caragea (2021)). Moreover, pruning can improve model robustness according to the following studies (Jordão and Pedrini (2021), Pang et al (2021), Hendrycks and Dietterich (2019) ).…”

SensitivAlert: Image Sensitivity Prediction in Online Social Networks Using Transformer-Based Deep Learning Models

“…They used SVM for the classification into private or public classes. In contrast, Zhao et al (2021) fine-tuned BERT to model images based on their user tags. (Zhao et al 2022) investigated the image privacy prediction by fine-tuning ResNet pre-trained models on object and scene recognition on PrivacyAlert.…”