Narrative Cognition in Mixed Reality Systems: Towards an Empirical Framework

Bruni, Luis Emilio; Dini, Hossein; Simonetti, Aline

doi:10.1007/978-3-030-77599-5_1

Cited by 9 publications

(9 citation statements)

References 76 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We showed that small differences in narrativity levels (i.e., between HL and LL) have effects on ISC and engagement similar to the effects of more evident differences between such levels (e.g., scrambled vs. non-scrambled). These findings are a step toward narrative comprehension, especially when considering narrative as a scalar property [35].…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Higher levels of narrativity lead to similar patterns of posterior EEG activity across individuals

Dini

Simonetti

Bigné

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

The focus of cognitive and psychological approaches to narrative has not so much been on the elucidation of important aspects of narrative, but rather on using narratives as tools for the investigation of higher order cognitive processes elicited by narratives (e.g., understanding, empathy, etc.). In this study, we work toward a scalar model of narrativity, which can provide testable criteria for selecting and classifying communication forms in their level of narrativity. We investigated whether being exposed to videos with different levels of narrativity modulates shared neural responses, measured by inter-subject correlation, and engagement levels. Thirty-two participants watched video advertisements with high-level and low-level of narrativity while their neural responses were measured through electroencephalogram. Additionally, participants engagement levels were calculated based on the composite of their self-reported attention and immersion scores. Results demonstrated that both calculated inter-subject correlation and engagement scores for high-level video ads were significantly higher than those for low-level, suggesting that narrativity levels modulate inter-subject correlation and engagement. We believe that these findings are a step toward the elucidation of the viewers way of processing and understanding a given communication artifact as a function of the narrative qualities expressed by the level of narrativity.

show abstract

Section: Discussionmentioning

confidence: 99%

“…However, the underlying reason for differences in ISC between narratives with some degree of narrativity level did not reflect differences in perceived engagement; rather, it seemed related to perspective taking. These findings are a step toward narrative comprehension, especially when considering narrative as a scalar property [36].…”

Section: Discussionmentioning

confidence: 99%

Higher levels of narrativity lead to similar patterns of posterior EEG activity across individuals

Dini

Simonetti

Bigné

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…All this means that the evolution of XR will inevitably require a better understanding of the technology as a medium, highlighting the nontechnological aspects of experience, as well as the eminent role of narrative cognition as a fundamental human cognitive mode for organizing and mediating experience. The combination of XR technologies, cognitive sciences, advanced biometric signals processing, and storytelling techniques will allow us to investigate the ways we interact cognitively with stories and the effects of the narratives on the human emotional-cognitive system [8]. Given the many different domains of applications for interactive narratives in XR and its specific requirements in terms of multimodality and embodied cognition, research in narrative cognition will be necessary to aid the development of communication strategies to use effectively the narrative mode in order to harness its potential beneficial applications in XR media.…”

Section: Future Perspectivesmentioning

confidence: 99%

“…This turn has many implications for the development of XR; both as a medium for narrative representation and for XR experiences in general. In other words, XR technologies are becoming privileged media for new kinds of narrative representation, and at the same time, the "narrative cognitive mode" is becoming a suitable framework for investigating user experience in immersive simulations [8].…”

Section: Introductionmentioning

confidence: 99%

Digital Narratives in Extended Realities

Bruni¹,

Kadastik²,

Pedersen³

et al. 2022

Roadmapping Extended Reality

View full text Add to dashboard Cite

XR technologies are increasingly considered as expressive media with special qualities for narrative representation. In this chapter, we chart some of the major challenges inherent in the implementation of digital narratives in XR. For this purpose, we consider the intersections between different research domains like classical and post-structural narratology, cognitive narratology, literary studies, computer science, game research, cognitive science, and particularly interactive digital narratives (IDN) and storytelling. As with IDN, many of the challenges of implementing narratives in XR have to do with the trade-offs between interactivity and immersion. In this direction, we aim to contribute to the bridging between the IDN and XR research domains. We elaborate on specific issues in IDN that may require particular care in XR, such as agency, embodiment, narration, "point of view technologies, " artificial agents, and AI. Additionally, we provide examples of different application domains in games, serious games, cultural experiences, healthcare, and immersive journalism. To conclude, we highlight the eminent role of narrative cognition in XR and suggest a roadmap for combining XR technologies, cognitive sciences, advanced biometric signals processing, and storytelling techniques for investigating ways we interact cognitively with stories and the effects of XR narratives on the human emotional-cognitive system.

show abstract

“…In fact, the “nexus of narrative and mind” (Herman, 2009, p.30) has been the concern of cognitive narratology for the past two decades. While acknowledging the challenges of exploring the field of narrative cognition (Bruni et al, 2021), this study investigates: (a) how the engagement levels fluctuate across a dramatic arc in line with a narrative; (b) whether it is possible to predict this engagement from neural activity; and (c) in order to further examine narrative comprehension, the idiosyncratic effects of each phase of the dramatic arc on brain responses as well as the relation between engagement and brain responses are explored. We use an explorative approach to fulfill these goals.…”

Section: Introductionmentioning

confidence: 99%

Exploring the Neural Processes behind Narrative Engagement: An EEG Study

Dini

Simonetti

Bruni

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Past cognitive neuroscience studies using naturalistic stimuli have considered narratives holistically and focused on cognitive processes. In this study, we incorporated the narrative structure the dramatic arc as an object of investigation, to examine how engagement levels fluctuate across a narrative-aligned dramatic arc. We explored the possibility of predicting self-reported engagement ratings from neural activity and investigated the idiosyncratic effects of each phase of the dramatic arc on brain responses as well as the relationship between engagement and brain responses. We presented a movie excerpt following the six-phase narrative arc structure to female and male participants while collecting EEG signals. We then asked this group of participants to recall the excerpt, another group to segment the video based on the dramatic arc model, and a third to rate their engagement levels while watching the movie. The results showed that the self-reported engagement ratings followed the pattern of the narrative dramatic arc. Moreover, whilst EEG amplitude could not predict group-averaged engagement ratings, other features comprising dynamic inter-subject correlation, dynamic functional connectivity patterns and graph features were able to achieve this. Furthermore, neural activity in the last two phases of the dramatic arc significantly predicted engagement patterns. This study is the first to explore the cognitive processes behind the dramatic arc and its phases. By demonstrating how neural activity predicts self-reported engagement, which itself aligns with the narrative structure, this study provides insights on the interrelationships between narrative structure, neural responses, and viewer engagement.

show abstract

Narrative Cognition in Mixed Reality Systems: Towards an Empirical Framework

Cited by 9 publications

References 76 publications

Higher levels of narrativity lead to similar patterns of posterior EEG activity across individuals

Higher levels of narrativity lead to similar patterns of posterior EEG activity across individuals

Digital Narratives in Extended Realities

Exploring the Neural Processes behind Narrative Engagement: An EEG Study

Contact Info

Product

Resources

About