Interactive image driven sound

Brennecke, Angela; Traber, Markus; Stimberg, Simon; Stockleben, Bjoern

doi:10.1145/3404983.3410004

Cited by 2 publications

(3 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The study in [9] described an interactive audiovisual synthesis application by analyzing visual changes in the image sequences and map the results to musical scales. The proposed Interactive Visual Audio (IVA) system reads the camera-captured images, extracts the visual features and then maps them to musical sounds.…”

Section: Related Workmentioning

confidence: 99%

“…DanceTalk can simply transform the dance of the performers into music. Specifically, we may replace the MediaPipe component by the IVA mechanism proposed in [9], and then fork the Join links to connect to a music playing output device as illustrated in Figure 16 (1). A video about a CATtalk music playing device is given in [25].…”

Section: Interactions With the Dancersmentioning

confidence: 99%

“…(9)) is connected to Position-O of D-mirror (Figure13(11)) and Position-O of S-Mirror (Figure 13 (13)) in Join 1. Similarly, Join 2 connects the Pattern DFs.…”

mentioning

confidence: 99%

See 2 more Smart Citations

CATtalk: An IoT-Based Interactive Art Development Platform

2022

View full text Add to dashboard Cite

Interactive art has been significant advanced by the Internet of Things (IoT) and cyber physical interaction technologies, which enables the participants to engage with the art devices. Several tools and platforms have been proposed to create the art devices. However, the interactive artworks are typically developed with these art devices in ad hoc approaches, and the artists need to spend significant programming efforts to integrate the art devices. This paper proposes CATtalk, a platform to create and maintain interactive artworks. The novel idea is to treat all art devices in an interactive artwork as IoT devices that can be transparently reused by reconfiguration in CATtalk. Therefore, the artworks developed independently by individual artists can be quickly integrated to create new interactive applications. Through CATtalk's no-code and low-code mechanisms, the artists can manipulate CATtalk with little or no programing efforts. CATtalk offers a built-in mechanism so that any person in the audience can play with an interactive artwork with his/her smartphone. We also conduct analytic analysis, simulation and measurements to ensure that the interactive art performance in cross-country remote stages are not affected by the communications delays. In our measurements, the average local and remote communication delays are about 0.01 and 0.05 seconds, respectively. If the art performance is designed such that the average delay between two actions of a local (remote) performer is longer than 0.1 seconds, then the probability of out-of-sequence actions is less than 0.01%. That is, the local dancer should perform slower than the remote dancer. Such delay analysis for remote interactive art performance has not been conducted in the literature.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Interactions With the Dancersmentioning

confidence: 99%

See 1 more Smart Citation

CATtalk: An IoT-Based Interactive Art Development Platform

2022

View full text Add to dashboard Cite

show abstract

The Virtual Theremin: Designing an Interactive Digital Music Instrument for Film Scene Scoring

Usabaev

Eschenbacher

Brennecke

2022

I-Com

View full text Add to dashboard Cite

This paper presents a first prototype of a virtual Theremin instrument for accompanying film scenes with sound. The virtual Theremin is implemented as a hybrid application for the web. Sound control is achieved by capturing user gestures with a webcam and mapping the gestures to the corresponding virtual Theremin parameters pitch and volume. Different sound types can be selected. The application’s underlying research is part of the multi-modal digital heritage project KOLLISIONEN which targets to open up the private archive of the Russian film maker Sergej Eisenstein to a broader public in digital form. Eisenstein, a film theorist and pioneer of film montage, was particularly intrigued by the Theremin as an instrument for film sound design. The virtual Theremin presented here is therefore linked to a film scene from the 1929 Soviet drama “The General Line” by Sergej Eisenstein which was never set to music originally. In its first implementation state, the application connects music interaction design with digital heritage in a modular, flexible and playful way and uses contemporary web technologies to enable easy operation and the greatest possible accessibility.

show abstract

Interactive image driven sound

Cited by 2 publications

References 6 publications

CATtalk: An IoT-Based Interactive Art Development Platform

CATtalk: An IoT-Based Interactive Art Development Platform

The Virtual Theremin: Designing an Interactive Digital Music Instrument for Film Scene Scoring

Contact Info

Product

Resources

About