Constructing the Immersive Interactive Sonification Platform (iISoP)

Abstract

For decades, researchers have spurred research on sonification, the use of non-speech audio to convey information [1]. With 'interaction' and 'user experience' being pervasive, interactive sonification [2], an emerging interdisciplinary area, has been introduced and its role and importance have rapidly increased in the auditory display community. From this background, we have devised a novel platform, "iISoP" (immersive Interactive Sonification Platform) for location, movement, and gesture-based interactive sonification research, by leveraging the existing Immersive Visualization Studio (IVS) at Michigan Tech. Projects in each developmental phase and planned research are discussed with a focus on "design research" and "interactivity".

From rituals to magic: Interactive art and HCI of the past, present, and future

“…Finally, designing and implementing technologies are creative processes per se, but HCI researchers also contribute to art and design by devising creativity-support tools (e.g., Shneiderman, 2017) as well as by providing design research platforms for artists (e.g., Jeon, Smith, Walker, & Kuhl, 2014). Table 1.…”

“…The immersive Interactive Sonification Platform (iISoP) (Jeon et al, 2014) builds upon these previous efforts to develop an interactive virtual environment for art-driven performance research. Due to its modular nature, the iISoP is capable of multimodal input (outside-in Vicon cameras, wearable inside-in gesture and physiological sensors) and output (auditory and visual).…”

“…Figure 1. While a performing artist is drawing on a paper canvas, a new version of his drawing based on his movements is displayed on a digital canvas at iISoP (Jeon et al, 2014).…”

“…Figure 3. An animal is playing the role of an agent who contributes to the entire artwork at iISoP (Jeon et al, 2014). also collaborate with each other to conduct new experiments to test, validate, and refine the platform.…”

Improving everyday computing tasks with head-mounted displays

“…iISoP was initially developed through a series of three phases of increasing complexity [24]. The first phase was designed as an interactive map which would play recordings in different languages based on the user's position.…”