Seeing What I Did (Not): Cerebral and Behavioral Effects of Agency and Perspective on Episodic Memory Re-activation

Abstract

Intuitively, we assume that we remember episodes better when we actively participated in them and were not mere observers. Independently of this, we can recall episodes from either the first-person perspective (1pp) or the third-person perspective (3pp). In this functional magnetic resonance imaging (fMRI) study, we tested whether agency and perspective modulate neural activity during memory retrieval and subsequently enhance memory performance. Subjects encoded a set of different episodes by either imitating or only observing videos that showed short toy stories. A week later, we conducted fMRI and cued episodic retrieval by presenting the original videos, or slightly modified versions thereof, from 1pp or from 3pp. The hippocampal formation was sensitive to self-performed vs. only observed actions only when there was an episodic mismatch. In a post-fMRI memory test a history of self-performance did not improve behavioral memory performance. However, modified videos were often (falsely) accepted as showing truly experienced episodes when: (i) they were already presented in this modified version during fMRI or (ii) they were presented in their original form during fMRI but from 3pp. While the overall effect of modification was strong, the effects of perspective and agency were more subtle. Together, our findings demonstrate that self-performance and self-perspective modulate the strength of a memory trace in different ways. Even when memory performance remains the same for different agentive states, the brain is capable of detecting mismatching information. Re-experiencing the latter impairs memory performance as well as retrieving encoded episodes from 3pp.

Constructing a Wider View on Memory

“…In recent work investigating the role of agency in episodic remembering, Jainta et al (2022) reveal findings that run counter to McCarroll's (2018) enrichment proposal. Their study investigated the level of prediction error in episodic recall.…”

“…Though remembering-from-the-outside is the most advanced theory of memory perspective on offer, it faces some issues. Notably, emerging empirical evidence suggests that information about one's visual perspective might not be encoded in memory traces at all (Jainta et al 2022). We will review these and other findings in the next section.…”

“…The researchers observed that contrasting novel videos with previously encountered episodes yielded a strong effect for first person versus third person. They hypothesize that viewing novel videos from the first person formed a sharp contrast to what the participants expected from their own perspective and therefore leads to a larger prediction error (i.e., a larger BOLD signal in fMRI, Jainta et al 2022).…”

Agency effects on the binding of event elements in episodic memory

“…This research extends previous findings on effects of agency in relation to memory and cognition. Although agency has been found to influence visual attention ( Huffman & Brockmole, 2020 ; Wen & Haggard, 2018 ), object vision ( Haxby et al, 2020 ; Thorat et al, 2019 ), face perception ( Walker & Keller, 2019 ), autobiographical memory ( Woike et al, 1999 ; Woike & Polo, 2001 ), hippocampal responses to expectation violations in episodic memory ( Jainta et al, 2022 ), and memory performance ( Hon & Yeo, 2021 ; Hornstein & Mulligan, 2001 ), this research suggests that effects of agency, at least as part of the stimulus, on cognition do not or only weakly extend to the formation of memory representations in episodic memory by influencing binding processes. Importantly, we focused on agency as part of an (external) stimulus, whereas the majority of previous research has focused on the agency of the participant (e.g., see the enactment effect, Engelkamp, 1986 ; Hornstein & Mulligan, 2001 ; Roberts et al, 2022 ).…”

“…Agentic-motivated individuals reported memories that were more congruent with their motives and structured them using more differentiation. Jainta et al (2022) found effects of agency during encoding on episodic memory. When encoding different episodes by imitating or merely observing videos showing short stories, participants exhibited stronger hippocampal responses to expectation violations when they were actors rather than observers in the episode.…”