Generative adversarial networks unlock new methods for cognitive science

Goetschalckx, Lore; Andonian, Alex; Wagemans, Johan

doi:10.1016/j.tics.2021.06.006

Cited by 16 publications

(11 citation statements)

References 82 publications

(115 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, such direct engineering of the generative features restricts the models of each stimulus category to the imagination of human engineers. A promising alternative uses generative adversarial networks (GANs) [68] which can produce arguably more realistic images. However, although such indirect engineering could expand generative parameters beyond human imagination, GANs are more difficult to control and therefore so far cannot deliver controlled 3D models of stimulus categories as computer graphics does (but note progress on structuring latent spaces [69][70][71]).…”

Section: Trends Trends In In Cognitivementioning

confidence: 99%

Degrees of algorithmic equivalence between the brain and its DNN models

Schyns¹,

Snoek²,

Daube³

2022

Trends in Cognitive Sciences

View full text Add to dashboard Cite

Section: Trends Trends In In Cognitivementioning

confidence: 99%

Degrees of algorithmic equivalence between the brain and its DNN models

Schyns¹,

Snoek²,

Daube³

2022

Trends in Cognitive Sciences

View full text Add to dashboard Cite

“…We investigated the trend of visual changes with each layer in our material (Fig 6) in which we used StyleGAN2. It shows the clear trends that the Coarse layers (1-4) correspond to the shape and posture, the Middle layers (5-8) correspond to physiognomy, and the Fine layers (9)(10)(11)(12)(13)(14)(15)(16)(17)(18) correspond to the color and texture of the face image. The fact that the Coarse and Middle layers have latent image features that bring about greater visual changes is consistent with the result that the RMSE in CP was lower in those layers.…”

Section: Discussionmentioning

confidence: 99%

“…Another study took advantage of GANs to continuously generate realistic images to examine the correlation between generated images and mental representations of visual experiences in terms of perceptual similarity and memory properties, and reported similarities with previous studies obtained with simpler visual stimuli 17 . In addition to improving the quality of the images generated by GANs, they presented a new experimental approach to cognitive science by generating continuously changing visual stimuli in a latent space composed of meaningful information structures 18 . Previous studies have revealed the relationship between human perception and cognitive response to the distance in stimulus space with continuously generated visual stimulus such as colors 19,20 .…”

Section: Introductionmentioning

confidence: 99%

Human Latent Metrics: Perceptual and Cognitive Response Corresponds to Distance in GAN Latent Space

Kasahara

Ienaga

Shimizu

et al. 2022

Preprint

View full text Add to dashboard Cite

Generative adversarial networks (GANs) generate high-dimensional vector spaces (latent spaces) that can interchangeably represent vectors as images. Advancements have extended their ability to computationally generate images indistinguishable from real images such as faces, and more importantly, manipulate images using their inherit vector values in the latent space. This interchangeability of latent vector has the potential to calculate not only distance in the latent space, but also human perceptual and cognitive distance toward images, i.e., how humans perceive and recognize images. However, it is still unclear how the distance in the latent space corresponds to human perception and cognition. Our studies investigated the correspondence between latent vectors and human perception or cognition through psycho-visual experiments that manipulates the latent vectors of face images. In the perception study, a change perception (CP) task was utilized to examine whether participants could perceive visual changes in face images before and after moving an arbitrary distance in the latent space. In the cognition study, a face cognition (FC) task was utilized to examine whether the participants could recognize a face as the same, even after moving an arbitrary distance in the latent space. The results showed that CP and cognition for face images clearly correlates to the distance in the latent space, which can be modeled with a logistic function. We also investigated how the internal layered structure of the latent space correlates to human response by calculating the regression residual error in each layer. As a result, we observed different residual error trends pertaining to CP and FC. Our experiments show that the distance between face images in the latent space corresponds to human perception and cognition for visual changes in face imagery, and additionally indicates that perception and cognition correspond with the latent space differently. By utilizing our methodology, it will be possible to interchangeably convert between the distance in the latent space and the metric of human perception and cognition, potentially leading to image processing that better reflects human perception and cognition.

show abstract

“…Modern systems such as generative adversarial networks 65 and derivatives of the classical variational autoencoders (VAEs) such as vector-quantized VAEs 66 , 67 and nouveau VAEs, 68 which can be trained on large, naturalistic face databases, can synthesize tantalizingly realistic faces, complete with hair, opening up an interesting avenue for future research and applications. 69 , 70 , 71 , 72 , 73 However, understanding and disentangling their latent spaces remains challenging. 74 , 75 …”

Section: Discussionmentioning

confidence: 99%