Synthetic Soundscapes with Natural Grains

Abstract

We present a technique to facilitate the creation of constantly changing, randomized audio streams from samples of source material. A core motivation is to make it easier to quickly create soundscapes for virtual environments and other scenarios where long streams of audio are used. While mostly in the background, these streams are vital for the creation of mood and realism in these types of applications. Our approach is to extract the component parts of sampled audio signals, and use them to resynthesize a continuous audio stream of indeterminate length. An automatic segmentation algorithm involving wavelets is used to split the input signal into syllable-like audio segments that we call "natural grains." For each grain, a table of similarity between it and all the other grains is constructed. The grains are then output in a continuous stream, with the next grain being chosen from among those other grains which best follow from it. Using this sampling-resynthesis technique, we can construct an infinite number of variations on the original signal with a minimum amount of interaction. An interface for the manipulation and playback of several of these streams is provided to facilitate building complex audio environments, and is made available for online experimentation at www.cs.ubc.ca/labs/lci/ naturalgrains/.

Kilo Hōkū—Experiencing Hawaiian, Non-Instrument Open Ocean Navigation through Virtual Reality

“…No background music is present in the simulation in order to provide as much immersion as possible. By creating a natural, 3D soundscape that the user would expect in this situation, immersion is enhanced (Hoskinson & Pai, 2007;Lumbreras & Ramírez, 2010).…”

Rhythmic Concatenative Synthesis for Electronic Music: Techniques, Implementation, and Evaluation

“…Its interface features knobs and sliders for interactively controlling gain and parameters of multiple samples. To evaluate the platform they conducted an additional study (Hoskinson and Pai 2007) to reveal whether listening subjects found the concatenated sequences convincing compared with genuinely recorded soundscapes.…”

Re-texturing the sonic environment

“…Hoskinson [8,9] uses a segmentation algorithm based on research in speech-processing and describes grain transitions using a Markov model, while Lu et al [11] and Parker and Behm [15] model and recreate textures by inferring recurrent patterns.…”