Latent Space Smoothing for Individually Fair Representations

Abstract

Fair representation learning encodes user data to ensure fairness and utility, regardless of the downstream application. However, learning individually fair representations, i.e., guaranteeing that similar individuals are treated similarly, remains challenging in high-dimensional settings such as computer vision. In this work, we introduce LASSI, the first representation learning method for certifying individual fairness of high-dimensional data. Our key insight is to leverage recent advances in generative modeling to capture the set of similar individuals in the generative latent space. This allows learning individually fair representations where similar individuals are mapped close together, by using adversarial training to minimize the distance between their representations. Finally, we employ randomized smoothing to provably map similar individuals close together, in turn ensuring that local robustness verification of the downstream application results in end-to-end fairness certification. Our experimental evaluation on challenging real-world image data demonstrates that our method increases certified individual fairness by up to 60%, without significantly affecting task utility.

Toward Identity-Invariant Facial Expression Recognition: Disentangled Representation via Mutual Information Perspective

“…Next, the generation results were verified through user study that rates the realism of each image. This experiment was designed because ID classifiers for ID-shifting [56], [57] are not normally available. In Table 1, EIF improved by 77% and 83% in realism score than ELIM and CAF, respectively.…”

Certifying Some Distributional Fairness with Subpopulation Decomposition

“…Remark. In the literature, the concepts of fairness are usually directly defined at the model prediction level, where the criterion is whether the model prediction is fair against individual attribute changes [36,33,47] or fair at population level [51]. In this work, to certify the fairness of model prediction, we define a fairness constrained distribution on which we will certify the model prediction (e.g., bound the prediction error), rather than relying on the empirical fairness evaluation.…”

“…Thus, when H(P s,y , Q s,y ) ≤ γs,y and sup (X,Y )∈X ×Y (h θ (X), Y ) ≤ M , given that is a non-negative loss by Section 2, we can see Equation (33), i.e., the expression in Thm. 3's statement, upper bounds Problem 1, i.e., provides a fairness certificate for Problem 1.…”

“…In addition to existing approaches that evaluate fairness, it is important and challenging to provide certification for ML fairness. Recent studies have explored the certified fair representation of ML [36,4,33]. However, there lacks certified fairness on the predictions of an end-to-end ML model trained on an arbitrary data distribution.…”

“…Different approaches are proposed to analyze group fairness via static analysis [43], interactive computation [38], and probabilistic approaches [1,9,6]. In addition, there is a line of work trying to certify the fair representation [36,4,33]. Our certified fairness differs from existing work from three perspectives: 1) we provide fairness certification considering the end-to-end model performance instead of the representation level, 2) we define and certify fairness based on a fairness constrained distribution which implies other fairness notions, and 3) our certified fairness can be computed for any black-box models trained on an arbitrary given data distribution.…”