Automatic design of sound synthesizers as pure data patches using coevolutionary mixed-typed cartesian genetic programming

Abstract

A sound synthesizer can be defined as a program that takes a few input parameters and returns a sound. The general sound synthesis problem could then be formulated as: given a sound (or a set of sounds) what program and set of input parameters can generate that sound (set of sounds)? We propose a novel approach to tackle this problem in which we represent sound synthesizers using Pure Data (Pd), a graphic programming language for digital signal processing. We search the space of possible sound synthesizers using Coevolutionary Mixed-typed Cartesian Genetic Programming (MT-CGP), and the set of input parameters using a standard Genetic Algorithm (GA). The proposed algorithm coevolves a population of MT-CGP graphs, representing the functional forms of synthesizers, and a population of GA chromosomes, representing their inputs parameters. A fitness function based on the Mel-frequency Cepstral Coefficients (MFCC) evaluates the distance between the target and produced sounds. Our approach is capable of suggesting novel functional forms and input parameters, suitable to approximate a given target sound (and we hope in future iterations a set of sounds). Since the resulting synthesizers are presented as Pd patches, the user can experiment, interact with, and reuse them.

Play it Again: Evolved Audio Effects and Synthesizer Programming

“…Further examination of performance versus human generated presets and sound samples is required to determine efficacy in more realistic applications. Another common approach and interest in GA based synthesizer programming is to employ targets originating from other sources (either other synthesizers or natural, sampled sounds) [8]. Recreating presets from samples of analog synthesizers, or acoustic instruments, can pose additional challenges and remains to be examined in the context of this system.…”

“…Selection Timbre matching is the primary problem being faced and the fitness function takes a standard approach [8,16], using the first 13 cepstra of a windowed MFCC analysis. This has the advantage of representing the spectral energy in a consistent and comparable way, capturing both envelope (amplitude) and tone generator (spectral weighting) characteristics.…”

“…Other work [8,16] employs Mel-Frequency Cepstrum Coefficient (MFCC) analysis to capture timbrel information and inform fitness decisions. MFCCs are standard in natural language processing and are shown to work effectively in capturing both spectral and amplitude aspects of time varying sounds.…”

“…Attempting to design the whole synthesizer based on a given target sound is also being examined [1,8], but presents a much more complicated domain compared to programming a given synthesizer. Optimizing the synthesis model or discovering completely new synthesis algorithms to accurately reproduce a given target sound is showing promise but has significant research ahead.…”

Interactively Evolving Compositional Sound Synthesis Networks

“…For instance, with a given synthesizer, parameters can be evolved to transform a synthesized waveform into target waveforms [12,20,16]. Similarly, other methods evolve these waveforms directly [17,19,1,7]. While these approaches can potentially help musicians to create synthesized sounds for music production, they rely on matching existing sounds rather than exploring the space of entirely new sounds.…”