The Self in the Mind’s Eye: Revealing How We Truly See Ourselves Through Reverse Correlation

Abstract

Is there a way to visually depict the image people “see” of themselves in their minds’ eyes? And if so, what can these mental images tell us about ourselves? We used a computational reverse-correlation technique to explore individuals’ mental “self-portraits” of their faces and body shapes in an unbiased, data-driven way (total N = 116 adults). Self-portraits were similar to individuals’ real faces but, importantly, also contained clues to each person’s self-reported personality traits, which were reliably detected by external observers. Furthermore, people with higher social self-esteem produced more true-to-life self-portraits. Unlike face portraits, body portraits had negligible relationships with individuals’ actual body shape, but as with faces, they were influenced by people’s beliefs and emotions. We show how psychological beliefs and attitudes about oneself bias the perceptual representation of one’s appearance and provide a unique window into the internal mental self-representation—findings that have important implications for mental health and visual culture.

Distortion of mental body representations

“…Two recent studies [101,102] measured individual 'self portraits' using reverse correlation, a data-driven method for reconstructing internal representations of categories. On each trial, the participant sees two images of a base face with random noise added, and judges which looks most like themselves.…”

Face yourself: The social neuroscience of mirror gazing

“…Interestingly, while recognition of self-face correlated with the activity of the action MNS, changes in embarrassment were co-varying with activity of both the MNS and the mentalizing system (Morita et al, 2008). Another recent study (Maister et al, 2021) inquired into the pictorial visual representation of individuals and compared it with various selfconstrual index, and showed that the valence of individuals selfrepresentation correlated with self-attributed visual features.…”

The psychophysics of empathy: Using reverse-correlation to quantify the overlap between self & other representations of emotional expressions

“…In [51], the perceived valence of the participants' representations of their own face correlated with their self-esteem and extraversion and, negatively, with social anxiety. In [47], the accuracy of the representations (i.e. how much they matched participants' real portraits) also correlated with participant's social self-esteem.…”

“…That procedure was used, e.g., to uncover subtle differences between the mental representations of reward, affiliation and dominance smiles [61], intense pain and pleasure [12] or, in the vocal domain, confidence and honesty [29]. Two recent studies [51,47] have used data-driven methods (namely, reverse-correlation or bubbles [55]) to investigate self-representations, by asking participants to rate how much manipulated photographs of a single composite face (the same for all participants) looked like their own face, and then asking a second set of participants to rate the visual properties of the obtained representations. In [51], the perceived valence of the participants' representations of their own face correlated with their self-esteem and extraversion and, negatively, with social anxiety.…”

“…While these studies suggest that data-driven methods can be used to investigate the content of self representations, the direct comparison, within the same observer, of judgements on both self and other stimuli remains difficult. First, because traditional reverse correlation randomizes stimuli in pixel-space [51,47], the obtained mental representations are quantified relatively to the base stimuli (e.g. how larger the eyes, or how thinner the chin) and their morphological features cannot be directly compared to representations obtained from manipulations of other photographs.…”