Shift-Life Interactive Art: Mixed-Reality Artificial Ecosystem Simulation

Abstract

This article presents a detailed design, development, and implementation of a Mixed Reality Art-Science collaboration project which was exhibited during Darwin’s bicentenary exhibition at Shrewsbury, England. As an artist-led project the concerns of the artist were paramount, and this article presents Shift-Life as part of an ongoing exploration into the parallels between the nonlinear human thinking process and computation using semantic association to link items into ideas, and ideas into holistic concepts. Our art explores perceptions and states of mind as we move our attention between the simulated world of the computer and the real world we inhabit, which means that any viewer engagement is participatory rather than passive. From a Mixed Reality point of view, the lead author intends to explore the convergence of the physical and virtual, therefore the formalization of the Mixed Reality system, focusing on the integration of artificial life, ecology, physical sensors, and participant interaction through an interface of physical props. It is common for digital media artists to allow viewers to activate a work either through a computer screen via direct keyboard or mouse manipulation, or through immersive means. For “Shift-Life” the artist was concerned with a direct “relational” approach where viewers would intuitively engage with the installation’s everyday objects, and with each other, to fully experience the piece. The Mixed Reality system is mediated via physical environmental sensors, which affect the virtual environment and autonomous agents, which in turn reacts and is expressed as virtual pixels projected onto a physical surface. The tangible hands-on interface proved to be instinctive, attractive, and informative on many levels, delivering a good example of collaboration between the arts and science.

A Classification Schema for Designing Augmented Reality Experiences

“…Alternate reality, on the other hand, provides a mixed reality experience that adds new narratives to the real-world setting (Deterding et al, 2011;Silva et al, 2016). The precise boundaries of these alternative realities rightly remain vague (Ch'ng et al, 2017). While we use AR as the primary term throughout this paper as it is often incarnated in highly varied forms, the classification schema is also able to provide greater resolving power in describing the diversity of mixed reality experiences.…”

“…This mix of physical and virtual can be perceived in different ways by participants, allowing potentially imaginative, abstract and participant rendered representations of reality (Avouris & Yiannoutsou, 2012; R. T. Azuma, 1997;Brederode et al, 2005;Cheok et al, 2002;Ch'ng et al, 2017;Collins et al, 2017;Dunleavy et al, 2009;Harley et al, 2016;Hoang & Cox, 2018;Kysela & Storkova, 2015;Nakevska et al, 2014;Roo & Hachet, 2017;Rouse et al, 2015).…”

“…Interaction involves a feedback loop where the participant provides input to the experience, and the output from the experience is reciprocally presented to the participant (Salen & Zimmerman, 2004;Steuer, 1992). The part of the feedback loop representing the actions provided by the participant can be achieved by using physical objects directly (Ch'ng et al, 2017), or re-imagined as particular tools (R. Azuma et al, 2001;Cheok et al, 2002) by manipulating their appearance with a visual overlay. The participant's body can be regarded as the physical source of actions (Hansen, 2012); for example, re-imagined as an asteroid participating in a planetary simulation (Lindgren & Moshell, 2011).…”

Future Application of Multisensory Mixed Reality in the Human Cyber—physical System

“…Various solutions to support MR environments are being developed and used across a wide range of industries. To comprehend better the evolution of technology and MR applications, the authors did a literature review [3,[5][6][7][8][9][10][11][12][13] from the Scopus database to build an MR ontology. We followed earlier research [3] in identifying manufacturing operations in which MR could enhance or facilitate technology or interaction devices.…”

“…It demonstrated that MR has already implemented numerous production procedures, from planning to maintenance [5][6][7]. In the context of MR technology, it can be split into input and output from the user's perspective [3,[8][9][10][11][12][13]. Both are crucial in the creation of immersive and intelligent systems to support human behaviour/nature.…”