Embracing deepfakes and AI‐generated images in neuroscience research

Becker, Casey; Laycock, Robin

doi:10.1111/ejn.16052

Cited by 8 publications

(5 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The cross-decoding performance might also have been impacted by the limited range of images in the perception task which might not fully cover the variability in the imagined visual contents. This could be remedied in future studies by generating a large image set based on the imagined prompts using text-to-image algorithms 45 . In addition, due to the relatively long trial duration, we only had 192 imagery trials of training data per participant, which further limited classifier performance.…”

Section: Discussionmentioning

confidence: 99%

Representations of imaginary scenes and their properties in cortical alpha activity

Stecher,

Kaiser

2024

Sci Rep

View full text Add to dashboard Cite

Imagining natural scenes enables us to engage with a myriad of simulated environments. How do our brains generate such complex mental images? Recent research suggests that cortical alpha activity carries information about individual objects during visual imagery. However, it remains unclear if more complex imagined contents such as natural scenes are similarly represented in alpha activity. Here, we answer this question by decoding the contents of imagined scenes from rhythmic cortical activity patterns. In an EEG experiment, participants imagined natural scenes based on detailed written descriptions, which conveyed four complementary scene properties: openness, naturalness, clutter level and brightness. By conducting classification analyses on EEG power patterns across neural frequencies, we were able to decode both individual imagined scenes as well as their properties from the alpha band, showing that also the contents of complex visual images are represented in alpha rhythms. A cross-classification analysis between alpha power patterns during the imagery task and during a perception task, in which participants were presented images of the described scenes, showed that scene representations in the alpha band are partly shared between imagery and late stages of perception. This suggests that alpha activity mediates the top-down re-activation of scene-related visual contents during imagery.

show abstract

Section: Discussionmentioning

confidence: 99%

Representations of imaginary scenes and their properties in cortical alpha activity

Stecher,

Kaiser

2024

Sci Rep

View full text Add to dashboard Cite

show abstract

“…The generated faces do not depict real persons, yet they are highly photorealistic and retain the resolution and diversity of the training set 36 . Studies show that humans are barely better than chance in discriminating similar AI-generated faces from real faces (although AI-generated faces may be seen as more trustworthy) [37][38][39][40] .…”

Section: Materials and Proceduresmentioning

confidence: 99%

Spontaneous Content of Impressions of Naturalistic Face Photographs

Nicolas,

Uddenberg,

Todorov

2024

Preprint

View full text Add to dashboard Cite

Social impressions from faces have been studied in psychology for over 100 years. These impressions are rapid, efficient, and consequential. Yet, disagreements about the content of face impressions remain. Across two studies (N = 4,526), we develop a taxonomy of spontaneous face impressions content by applying novel interdisciplinary methods from Artificial Intelligence text analysis to thousands of free-response descriptions of computer-generated faces. We identify a taxonomy of face impression dimensions, and describe their coverage, prevalence, directionality, and correlational structure. We characterize a diverse and nuanced taxonomy of content that, when compared to just the content that dominant low-dimensional models focus on, increases the coverage of spontaneous responses from about 50% to almost 100%. Our results describe general patterns of prevalence, indicating that dimensions from low-dimensional models (e.g., Sociability, Morality, Assertiveness) are highly prevalent, but that alternative dimensions such as Uniqueness and Health, among others, are also significantly prevalent in face impressions of naturalistic face photographs. Most dimensions show a positivity bias, and the correlational structure of the dimensions further supports the clustering of low-dimensional model’s content as separate from the expanded taxonomy dimensions. Finally, this expanded taxonomy improves predictions of general evaluations and decision making in various real-world relevant contexts. The derived taxonomy of spontaneous face impressions content is a foundation for further theoretical development and practical applications in an area central to human behavior.

show abstract

“…The ideal solution would be a dynamic, customizable face that retains the naturalistic movement of a video recording. Deepfake technology might provide such a solution (Becker & Laycock, 2023). Deepfakes are artificially generated videos that convincingly replace a person's face or alter their appearance using advanced machine learning techniques.…”

Section: Deepfakes In Face Perception Researchmentioning

confidence: 99%

“…Unlike dynamic morphs, high-quality deepfakes can capture naturalistic facial motion (Welker et al, 2020). Recently, deepfakes have garnered attention for their potential use in face perception research (Becker & Laycock, 2023). For example, Vijay et al (2021) used deepfake technology to manipulate the presence of eye-contact, smiling and nodding, thus experimentally isolating their impact.…”

Section: Deepfakes In Face Perception Researchmentioning

confidence: 99%

Can deepfakes be used to study emotion perception? A comparison of dynamic face stimuli

Becker,

Conduit,

Chouinard

et al. 2024

Behav Res

Self Cite

View full text Add to dashboard Cite

Video recordings accurately capture facial expression movements; however, they are difficult for face perception researchers to standardise and manipulate. For this reason, dynamic morphs of photographs are often used, despite their lack of naturalistic facial motion. This study aimed to investigate how humans perceive emotions from faces using real videos and two different approaches to artificially generating dynamic expressions – dynamic morphs, and AI-synthesised deepfakes. Our participants perceived dynamic morphed expressions as less intense when compared with videos (all emotions) and deepfakes (fearful, happy, sad). Videos and deepfakes were perceived similarly. Additionally, they perceived morphed happiness and sadness, but not morphed anger or fear, as less genuine than other formats. Our findings support previous research indicating that social responses to morphed emotions are not representative of those to video recordings. The findings also suggest that deepfakes may offer a more suitable standardized stimulus type compared to morphs. Additionally, qualitative data were collected from participants and analysed using ChatGPT, a large language model. ChatGPT successfully identified themes in the data consistent with those identified by an independent human researcher. According to this analysis, our participants perceived dynamic morphs as less natural compared with videos and deepfakes. That participants perceived deepfakes and videos similarly suggests that deepfakes effectively replicate natural facial movements, making them a promising alternative for face perception research. The study contributes to the growing body of research exploring the usefulness of generative artificial intelligence for advancing the study of human perception.

show abstract

Embracing deepfakes and AI‐generated images in neuroscience research

Cited by 8 publications

References 43 publications

Representations of imaginary scenes and their properties in cortical alpha activity

Representations of imaginary scenes and their properties in cortical alpha activity

Spontaneous Content of Impressions of Naturalistic Face Photographs

Can deepfakes be used to study emotion perception? A comparison of dynamic face stimuli

Contact Info

Product

Resources

About