Scenario: Co-evolution, shared autonomy and mixed reality

Abstract

This paper describes the Australian Research Council funded project Scenario, conducted at the iCinema Research Centre, University of New South Wales. Realized through interdisciplinary research, involving the domains of machine learning, interactive narrative and new media art, Scenario creates a mixed reality (MR) environment, surrounding the user within a 360-degree stereoscopic space, in which she can interact with digital characters that have a level of autonomy. Through its discussion of the aesthetics and technological architecture of Scenario the paper enters into an explanation of what is termed 'co-evolutionary' narrative, a function of the interactive relationship formed between a human user and an autonomous digital character. Understanding interaction as co-evolution the Scenario project enlists the philosophy of Gilles Deleuze and Manual DeLanda in order to propose interaction as a dynamic two-way process. The paper begins to unpack this theoretical framework for understanding interaction. INTRODUCTIONDigital aesthetics, in broad brush strokes, can be described as the critical reflection of the way digital technologies intersect with, or impact upon, the sensory processes by which we come to know the world. However, this is not to say that digital aesthetics are concerned merely with the 'representation' of the world by digital means. After all, new technologies are used most powerfully when they are applied as constructive, rather than 'representational' or decorative forces, expressing new ways of living in, understanding and interacting with the world. In this sense, we can say that at the heart of digital aesthetics is a concern with understanding the meaningful interactions between technological and human processes. Mixed Reality (MR) environments are progressively upscaling the everyday convergence of the technological and the human, dissolving the usual boundaries found at the interface of the screen, as digital images can enter the user's space. In these cases, and particularly where digital characters are given the capacity to act autonomously, interaction might be thought to embody an ecological relationship between users and digital characters, each affecting the way the other behaves and interprets the world. The relationship is ecological in the sense that interaction is a result of the interdependency between the human and the digital system, re-formatting the conventional aesthetic distinction set up between a beholder and a representational system [1]. Rather than a beholder encountering a linguistic or plastic 'representation', they, as agents in an interactive system, encounter other (digital) agents, such that both share levels of agency. Aesthetic considerations of such a model would focus on the philosophical concept of practical reasoning -as both human users and digital characters consider the way to act, or what to do, based on a relationship formed between internal and external forces -and a theory of narrative as central to the way we make meaning of the episodic ...

Postphenomenological Performance in Interactive Narrative

“…As Del Favero and Timothy Barker have highlighted, the origins of Scenario was to test out the formation of meaningful relationships between humans and technology by generating 'innovative research in the field of machine learning and artificial intelligence (AI), along with iCinema's ongoing research into immersive and interactive environments' (Favero & Barker, 2010). The result of this transaction between a human user and digital character in Scenario is what they term a coevolutionary narrative.…”

“…This is assisted by the dark ambient tones of these strange backdrops, designed to coerce a sense of uncanniness and foreboding in each participant's body. This is heightened as Del Favero and Barker write, by the way users experience 'the ambiguity of the sensory objects that surround [them]' (Favero & Barker, 2010) juxtaposed with sensations that are 'relatively familiar as [they] can see [their] own physical bodies and the bodies of the other users' (Favero & Barker, 2010).…”

“…In short, an assemblage is always constituted by the capacity for interaction between its parts. (Favero & Barker, 2010) In other words, it is not just the parts that make up an assemblage but also how they act, or how they could act, through the potentiality of their interaction to one another. 'It is the affect of the parts -as their capacity to act on one another -that matters, not their materiality, individual power or visual appearance' (Favero & Barker, 2010).…”

“…A machine is actualised within Scenario so that activity interrupts spectatorship, movement interrupts the flow of story, and movement from user to character and reciprocally from character to user interrupt and affect one another, which as Del Favero and Barker state, can be clearly seen: (Favero & Barker, 2010) he sensing technology of the interface has real observable effects on the user's movement. Users move more slowly around the space as the digital pace of the machine interrupts and conducts the flow of natural bodily rhythm.…”

Postphenomenological performance: bodily extensions in interactive art

“…This is assisted by the dark ambient tones of these strange backdrops, designed to coerce a sense of uncanniness and foreboding in each participant's body. This is heightened, as DelFavero and Barker (2010) explain, by the way users experience 'the ambiguity of the sensory objects thatsurround [them]' juxtaposed with sensations that are 'relatively familiar as [they] can see [their] own physical bodies and the bodies of the other users'. Within the third part of the artwork, the users are transported to a clearing in a forest.…”