Presentation in self-posted facial images can expose sexual orientation: Implications for research and privacy.

Abstract

Recent research has found that facial recognition algorithms can accurately classify people’s sexual orientations using naturalistic facial images, highlighting a severe risk to privacy. This article tests whether people of different sexual orientations presented themselves distinctively in photographs, and whether these distinctions revealed their sexual orientation. I found significant differences in self-presentation. For example, gay individuals were on average more likely to wear glasses compared to heterosexual individuals in images uploaded to the dating website. Gay men also uploaded brighter images compared to heterosexual men. To further test how some of these differences drove the classification of sexual orientation, I employed image augmentation or modification techniques. To evaluate whether the image background contributed to classifications, I progressively masked images until only a thin border of image background remained in each facial image. I found that even these pixels classified sexual orientations at rates significantly higher than random chance. I also blurred images, and found that merely three numbers representing the brightness of each color channel classified sexual orientations. These findings contribute to psychological research on sexual orientation by highlighting how people chose to present themselves differently on the dating website according to their sexual orientations, and how these distinctions were used by the algorithm to classify sexual orientations. The findings also expose a privacy risk as they suggest that do-it-yourself data-protection strategies, such as masking and blurring, cannot effectively prevent leakage of sexual orientation information. As consumers are not equipped to protect themselves, the burden of privacy protection should be shifted to companies and governments.

Facial recognition technology can expose political orientation from naturalistic facial images

“…Even more worryingly, facial recognition can infer a much broader range of personal attributes. Patents filed by organizations ranging from startups to Xerox (4,5) as well as a recent flurry of scientific papers show that facial recognition can accurately infer traits such as political orientation (6, 7), personality (8)(9)(10)(11), or sexual orientation (12)(13)(14).…”

“…Yet, these studies employed self-selected online profile pictures, which are replete with confounding variables that may be associated with-and thus reveal-political orientation (6,7). Those include self-presentation (e.g., makeup, facial hair style, and head orientation), facial expression, and image properties (e.g., resolution and sharpness) (12,14). Moreover, while many studies employed images of elected politicians, the differences between the faces of liberal and conservative politicians do not necessarily imply that the faces of their respective electorates also differ.…”

How you look is who you are: The appearance reveals character lay theory increases support for facial profiling.

“…Finally, whereas the physiognomic beliefs scale is focused only on the face, the appearance reveals character lay theory scale incorporates other aspects of appearance, such as body shape/size, hairstyle, and clothes. This is an important distinction as recent research suggests that the inference of specific character traits from appearance may also be influenced by factors other than the face (Gelman et al, 2018; Wang, in press). Thus, our construct taps a much more general belief than the physiognomic beliefs scale across three dimensions.…”

“…However, asking people to think about appearance more broadly would lead them to think of other salient features of appearance that are under the individual’s control (e.g., hair, clothes), and may not activate self-presentation concerns to the same extent. Third, despite the terminology, facial profiling algorithms do not just take individuals’ faces as their input—they take all aspects of the person’s appearance visible in the person’s photograph, including the face and several extrafacial characteristics (e.g., Wang, in press).…”

Facial recognition technology and human raters can predict political orientation from images of expressionless faces even when controlling for demographics and self-presentation.

“…Many previous studies relied on selfselected facial images (e.g., social media profile pictures) that contained confounding factors potentially related to political orientation. Such variables include self-presentation (e.g., makeup, facial hair style, and head orientation), facial expression, and image properties such as resolution and sharpness (D. Wang, 2022;Y. Wang & Kosinski, 2018).…”