Avui

Abstract

The combined use of sound and image has a rich history, from audiovisual artworks to research exploring the potential of data visualization and sonification. However, we lack standard tools or guidelines for audiovisual (AV) interaction design, particularly for live performance. We propose the AVUI (AudioVisual User Interface), where sound and image are used together in a cohesive way in the interface; and an enabling technology, the ofxAVUI toolkit. AVUI guidelines and ofx-AVUI were developed in a three-stage process, together with AV producers: 1) participatory design activities; 2) prototype development; 3) encapsulation of prototype as a plug-in, evaluation, and roll out. Best practices identified include: reconfigurable interfaces and mappings; object-oriented packaging of AV and UI; diverse sound visualization; flexible media manipulation and management. The toolkit and a mobile app developed using it have been released as open-source. Guidelines and toolkit demonstrate the potential of AVUI and offer designers a convenient framework for AV interaction design.

The Role of Live Visuals in Audience Understanding of Electronic Music Performances

“…The latter, because of its inherent tight and consistent mapping between drawing and audiovisual result. These two categories relate to our concept of Audiovisual User Interface (AVUI) [11], where "UI, audio and visualization are interconnected and integrated", leading to clear audiovisual responses to interaction.…”

“…Five projects were created in Gen.AV 1, and six in Gen.AV 2. In [11], we have reported on these hackathons, and subsequent evaluation of projects. This evaluation consisted of tests by other performers of the projects, over a period of around one week, followed by interviews to gather feedback.…”

“…Wat creates a chaotic audiovisual texture based on cellular automata, distributed in a rectangular 3D space. More information about all Gen.AV projects: (http://www.gen-av.org) and [10,11].…”

“…The popularization of powerful personal computers with multimedia capabilities in the 1990s empowered a new generation of visual and audiovisual performers to generate and manipulate digital content in real time, live [33]. Audiovisual (AV) artists often create their own systems for performance, with a DIY approach, using software and programming environments such as Max/MSP, PureData, Processing and openFrameworks [11]. Salter describes that this practice adopted "a long litany of names such as audiovisual performance, real-time video, live cinema, performance cinema, and VJ culture" [33].…”

From GUI to AVUI: Situating Audiovisual User Interfaces Within Human-Computer Interaction and Related Fields

“…Audiovisual relationships have been explored in the field of Human-Computer Interaction (HCI) to enhance the user experience and usability, across different application areas -such as accessibility in assistive displays [1], improvement of task accuracy in driving [2], alarms for surveillance activities [3] and enjoyability and performance in games [4,5]. To facilitate the implementation of congruent audiovisual feedback in interaction design, and its integration with user interfaces, we propose a new type of UI: AVUI (AudioVisual User Interface) [6]. The concept of AVUI links interaction, sound and image, building upon the concept of Graphical User Interface (GUI) by adding interconnected audio and visuals.…”