Designing Interactive Sonification for Live Aquarium Exhibits

Abstract

In response to the need for more accessible and engaging informal learning environments (ILEs), researchers have studied sonification for use in interpretation of live aquarium exhibits. The present work attempts to introduce more interactivity to the project's existing sonification work, which is expected to lead to more accessible and interactive learning opportunities for visitors, including children and people with vision impairment. In this interactive sonification environment, visitors can actively experience an exhibit by using tangible objects to mimic the movement of animals. Sonifications corresponding to their movement can be paired with real-time animal-based sonifications produced by the existing system to generate a musical fugue. In the current paper, we describe the system configurations, experiment results for optimal sonification parameters and interaction levels, and implications in terms of embodied interaction and interactive learning.

Constructing the Immersive Interactive Sonification Platform (iISoP)

“…To this end, it needs to evolve further with higher intellectual capability. Finally, the iISoP could be developed for virtual museums and galleries for better accessibility of audience with diverse disabilities and for enhanced user experience for all populations [11].…”

“…To illustrate, recent research on the aquarium scene sonification project [24] made informal learning environment (e.g., aquariums, zoos, etc.) more accessible to diverse populations and improved user experiences, simply by adding sonification of audience's gestures, which leads to a musical fugue with fish movement-based sonification [11,14]. Results suggest that interactive sonification could improve visitors' learning and overall experiences by letting them to play a more constructive and interactive role.…”

“…As an interactive sonification technique, parameter mapping [e.g., 4] has often been used, where data features are mapped onto acoustic attributes such as pitch, tempo, timbre, etc. Parameter mapping enables users to navigate through the (big) data, to adjust the mapping on prerecorded data, or to mold the sonification of data in real time [e.g., 10,11]. Another framework for interactive sonification is model-based sonification [4].…”

Animals Make Music: A Look at Non-Human Musical Expression

“…The projects we could find from this community are mostly combinations of a computer vision system tracking the animals, and a sound-generating system, which is controlled by the animals' movements. A common setup in this field is fish-based interfaces [24][25][26][27][28][29][30]. The video tracking system observes the fish in the aquarium, and the sounds are generated or processed by the generally random movements of the fish [24].…”

“…The Accessible Aquarium environment allows visitors to mimic the animals' movements by using tangible objects to tell when a new fish enters the viewing range, what kind of fish it is, and in what direction it's moving; or add sounds or melodies to a specific aquarium fish. Sonifications corresponding to the visitors' movements can be paired with real-time animal-based sonifications produced by the existing system to generate a musical composition of the visitors and the fish [26].…”