Interacting with 3D Reactive Widgets for Musical Performance

Journal of New Music Research

2020

Self Cite

As Virtual Reality headsets become accessible, more and more artistic applications are developed, including immersive musical instruments. 3D interaction techniques designed in the 3D User Interfaces research community, such as navigation, selection and manipulation techniques, open numerous opportunities for musical control. For example, navigation techniques such as teleportation, free walking/flying and path-planning enable different ways of accessing musical scores, scenes of spatialized sounds sources or even parameter spaces. Manipulation techniques provide novel gestures and metaphors, e.g. for drawing or sculpting sound entities. Finally, 3D selection techniques facilitate the interaction with complex visual structures which can represent hierarchical temporal structures, audio graphs, scores or parameter spaces. However, existing devices and techniques were developed mainly with a focus on efficiency, i.e. minimising error rate and task completion times. They were therefore not designed with the specifics of musical interaction in mind. In this paper, we review existing 3D interaction techniques and examine how they can be used for musical control, including the possibilities they open for instrument designers. We then propose a number of research directions to adapt and extend 3DUIs for musical expression KEYWORDS 3D user interfaces, Immersive virtual musical instruments, Virtual Reality, New interface for musical expression

Section: Dynamic/reactive Widgetsmentioning

confidence: 99%

Section: Virtual Pointing / Raymentioning

confidence: 99%

Section: Dedicated Input Devicesmentioning

confidence: 99%

See 1 more Smart Citation

3D interaction techniques for musical expression

Journal of New Music Research

2020

Self Cite

“…Manipulating a sound then amounts to manipulating the appearance of the associated widget, making the musical interaction more transparent to the user. We designed graphical modulation tools called Tunnels [3] for changing the appearance of the 3D reactive widgets, and consequently the sound they embed. Each Tunnel displays a scale of values, discrete or continuous, for one or several graphical parameters.…”

Section: First Circle: the Musicianmentioning

confidence: 99%

Spatial Interfaces and Interactive 3D Environments for Immersive Musical Performances

IEEE Comput. Grap. Appl.

Hachet²

2016

Self Cite

The power of interactive 3D graphics, immersive displays, and spatial interfaces is still under-explored in domains where the main target is to enhance creativity and emotional experiences. This article presents a set of work the attempts to extent the frontiers of music creation as well as the experience of audiences attending to digital performances. The goal is to connect sounds to interactive 3D graphics that musicians can interact with and the audience can observe.

“…In [3], we developed the idea of Tunnels, 3D graphical sliders which combine advantages of these solutions while providing new ones, such as the simultaneous modulation of several objects, multiple scales and an improved visual feedback. The purpose of these widgets is to allow users to efficiently modify the appearance, e.g.…”

Section: Introductionmentioning

confidence: 99%

Poster: Beyond the Tunnels: Advanced 3D graphical modulation

2012 IEEE Symposium on 3D User Interfaces (3DUI)

2012

Tunnels are 3D graphical sliders which modify the appearance of, or "modulate", one or several 3D objects simultaneously using these objects as cursors themselves. They rely on a strong visual feedback to provide multiple scales and parameters combinations which makes them appropriate for the use in immersive virtual musical instruments. In this paper we present applications of the Tunnels for collaborative modulation tasks, for avatars and weapons modifications in 3D video games, and for the control of interactive scenarios. We also propose several improvements to the Tunnels such as a method for storing them and a two-dimensional version called Grids.