Real Sound Synthesis for Interactive Applications

Cook, Perry R.

doi:10.1201/b19597

Cited by 190 publications

(94 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In general, the sound design of these approaches can be distinguished between sample based implementations using recordings of real-life footsteps and synthesized sounds. These are further classified into models aiming to simulate real-world walking sounds on different ground textures [11,12,18,19,30] and the design of abstract sounds for the purpose of providing additional information about gait characteristics to the recipient [21,22]. Bresin et al [7,8] analyzed the impact of acoustically augmented footsteps on walkers and investigated how far their emotional state was represented by the audio recordings of their walking movements.…”

Section: Sonification Of Force Sensor Datamentioning

confidence: 99%

SONIGait: a wireless instrumented insole device for real-time sonification of gait

Horsak

Dlapka

Iber

et al. 2016

J Multimodal User Interfaces

View full text Add to dashboard Cite

The treatment of gait disorders and impairments are major challenges in physical therapy. The broad and fast development in low-cost, miniaturized, and wireless sensing technologies supports the development of embedded and unobtrusive systems for robust gait-related data acquisition and analysis. Next to their applications as portable and lowcost diagnostic tools, such systems are also capable of use as feedback devices for retraining gait. The approach described within this article applies movement-based sonification of gait to foster motor learning. This article aims at presenting and evaluating a prototype of a pair of instrumented insoles for real-time sonification of gait (SONIGait) and to assess its immediate effects on spatio-temporal gait parameters. For this purpose, a convenience sample of six healthy males (age 35 ± 5 years, height 178 ± 4 cm, mass 78 ± 12 kg) and six healthy females (age 38 ± 7 years, height 166 ± 5 cm, mass: 63±8 kg) was recruited. They walked at a self-selected walking speed across a force distribution measurement system (FDM) to quantify spatio-temporal gait parameters during walking without and with five different types of sonification. The primary results from this pilot study revealed that participants exhibited decreased cadence (p < 0.01) and differences Results suggest that sonification has an effect on gait parameters, however further investigation and development is needed to understand its role as a tool for gait rehabilitation.

show abstract

Section: Sonification Of Force Sensor Datamentioning

confidence: 99%

SONIGait: a wireless instrumented insole device for real-time sonification of gait

Horsak

Dlapka

Iber

et al. 2016

J Multimodal User Interfaces

View full text Add to dashboard Cite

show abstract

“…On the other end of the spectrum, data-driven methods focus on using, or reproducing characteristics of, recorded data when the underlying physical systems are either too expensive to simulate or methods simply cannot produce convincing sound [Cook 2002;Picard et al 2009;Peltola et al 2007]. These techniques often utilize general synthesis algorithms, such as inverse-FFT synthesis [Rodet et al 1992;Marelli et al 2010], and additive and subtractive synthesis [Serra and Smith 1990].…”

Section: Related Workmentioning

confidence: 99%

Motion-driven concatenative synthesis of cloth sounds

James

Marschner

2012

TOG

View full text Add to dashboard Cite

Figure 1: Our data-driven approach to synthesizing cloth sounds is able to produce soundtracks for a wide range of common cloth animation scenarios. In this example, the familiar sounds of a windbreaker are synthesized as the character shadow boxes. AbstractWe present a practical data-driven method for automatically synthesizing plausible soundtracks for physics-based cloth animations running at graphics rates. Given a cloth animation, we analyze the deformations and use motion events to drive crumpling and friction sound models estimated from cloth measurements. We synthesize a low-quality sound signal, which is then used as a target signal for a concatenative sound synthesis (CSS) process. CSS selects a sequence of microsound units, very short segments, from a database of recorded cloth sounds, which best match the synthesized target sound in a low-dimensional feature-space after applying a handtuned warping function. The selected microsound units are concatenated together to produce the final cloth sound with minimal filtering. Our approach avoids expensive physics-based synthesis of cloth sound, instead relying on cloth recordings and our motiondriven CSS approach for realism. We demonstrate its effectiveness on a variety of cloth animations involving various materials and character motions, including first-person virtual clothing with binaural sound.

show abstract

“…A goal of this paper is the development of controls for independent manipulation of pitch and timbre of a sound source using a cortical sound representation that was introduced in [1] and used for assessment of speech intelligibility and for prediction of the cortical response to an arbitrary stimulus. We simulate the multiscale audio representation and processing believed to occur in the primate brain (supported by recent psychophysiological papers [2]), and while our sound decomposition is partially similar to existing pitch and timbre separation and sound morphing algorithms (in particular, MFCC decomposition algorithm in [3], sinusoid plus noise model and effects generated with it in [4], and parametric source models using LPC and physics-based synthesis in [5]), the neuromorphic framework allows to view the processing from a different perspective, supply supporting evidence to justify the procedure performed and tailor it to the way the human nervous system processes auditory information, and extend approach to include decomposition in time domain in addition to frequency.…”

Section: Introductionmentioning

confidence: 99%

“…In musical instrument synthesis, synthesizers often use sampled sound that have to be pitch-shifted to produce different notes [5] or generate a new instrument with the perceptual timbre lying inbetween two known instruments. Development of advanced auditory user interfaces requires mapping of arbitrary data streams into auditory percepts, and is commonly called "sonification" [6].…”

Section: Introductionmentioning

confidence: 99%

Pitch and timbre manipulations using cortical representation of sound

Zotkin

Shamma

et al. 2003

2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698)

View full text Add to dashboard Cite

The sound received at the ears is processed by humans using signalprocessing that separates the signal along intensity, pitch and timbre dimensions. Conventional Fourier-based signal processing, while endowed with fast algorithms, is unable to easily represent signal along these attributes. In this paper we use a recently proposed cortical representation to represent and manipulate sound. We briefly overview algorithms for obtaining, manipulating and inverting cortical representation of a sound and describe algorithms for manipulating signal pitch and timbre separately. The algorithms are first used to create sound of an instrument between a "guitar" and a "trumpet". Applications to creating maximally separable sounds in auditory user interfaces are discussed.Partial support of ONR grant N000140110571 is gratefully acknowledged.1. INTRODUCTION When a natural source such as a human voice or a musical instrument produces a sound, the resulting acoustic wave is generated by a time-varying excitation pattern of a possibly time-varying channel, and the sound characteristics depend both on the excitation signal and on the production system. The production system (e.g., human vocal tract, the guitar box, or the flute tube) has its own characteristic response; variation of the excitation parameters produces a sound signal that has different frequency components, but still retains perceptual characteristics of the uniqueness of the production instrument (identity of the person, type of instrumentpiano, violin, etc.) When one is asked to characterize this sound source using descriptions based on Fourier analysis one discovers that concepts such as frequency and amplitude are insufficient to explain the characteristics of the sound source. Human linguistic descriptions characterize the sound in terms of pitch and timbre.The perceived sound pitch is closely coupled with its harmonic structure. On the other hand, the timbre of the sound is defined broadly as everything other than the pitch, loudness, and the spatial location of the sound. For example, two musical instruments might have the same pitch if they play the same note, but it is their different timbre that allows us to distinguish between them. Specifically, the spectral envelope in frequency and the spectral envelope variations in time are related to the timbre percept. Most conventional techniques of sound manipulation result in simultaneous changes in both the pitch and timbre and cannot be used to assess the effects of the pitch and timbre dimensions independently. A goal of this paper is the development of controls for independent manipulation of pitch and timbre of a sound source using a cortical sound representation that was introduced in [1] and used for assessment of speech intelligibility and for prediction of the cortical response to an arbitrary stimulus. We simulate the multiscale audio representation and processing believed to occur in the primate brain (supported by recent psychophysiological papers [2]), and while our sound decomposition is p...

show abstract

Real Sound Synthesis for Interactive Applications

Cited by 190 publications

References 0 publications

SONIGait: a wireless instrumented insole device for real-time sonification of gait

SONIGait: a wireless instrumented insole device for real-time sonification of gait

Motion-driven concatenative synthesis of cloth sounds

Pitch and timbre manipulations using cortical representation of sound

Contact Info

Product

Resources

About