Forgetting Fast in Recommender Systems

Abstract

Users of a recommender system may want part of their data being deleted, not only from the data repository but also from the underlying machine learning model, for privacy or utility reasons. Such right-to-be-forgotten requests could be fulfilled by simply retraining the recommendation model from scratch, but that would be too slow and too expensive in practice. In this paper, we investigate fast machine unlearning techniques for recommender systems that can remove the effect of a small amount of training data from the recommendation model without incurring the full cost of retraining. A natural idea to speed this process up is to fine-tune the current recommendation model on the remaining training data instead of starting from a random initialization. This warm-start strategy indeed works for neural recommendation models using standard 1st-order neural network optimizers (like AdamW). However, we have found that even greater acceleration could be achieved by employing 2nd-order (Newton or quasi-Newton) optimization methods instead. To overcome the prohibitively high computational cost of 2nd-order optimizers, we propose a new recommendation unlearning approach AltEraser which divides the optimization problem of unlearning into many small tractable sub-problems.Extensive experiments on three real-world recommendation datasets show promising results of AltEraser in terms of consistency (forgetting thoroughness), accuracy (recommendation effectiveness), and efficiency (unlearning speed). To our knowledge, this work represents the first attempt at fast approximate machine unlearning for state-of-the-art neural recommendation models.

On the Effectiveness of Unlearning in Session-Based Recommendation

“…Unlearning can not only help to protect user privacy but also improve recommendation models through eliminating the effect of noisy data and misleading information [28]. [23] and [40] proposed to use fine-tuning and the alternative least square algorithm for unlearning acceleration. [5] and [22] extended the ideas of the SISA algorithm for collaborative filtering.…”

“…(2) 3.2.2 Unlearning effectiveness is not well defined. Existing recommendation unlearning methods [5,23] mainly investigated the trade-off between recommendation performance and unlearning efficiency. As pointed out in section 3.2.1, exact unlearning is hard to achieve in session-based recommendation.…”

“…Existing recommendation unlearning methods do not evaluate the unlearning effectiveness. Existing methods [5,23] mainly focus on the trade-off between recommendation performance and unlearning efficiency. However, seldom work conducted the evaluation regarding the unlearning effectiveness, i.e., to which extent the effect of unlearned samples is eliminated.…”