Computer-Aided Musical Orchestration Using an Artificial Immune System

Abstract

The aim of computer-aided musical orchestration is to find a combination of musical instrument sounds that approximates a target sound. The difficulty arises from the complexity of timbre perception and the combinatorial explosion of all possible instrument mixtures. The estimation of perceptual similarities between sounds requires a model capable of capturing the multidimensional perception of timbre, among other perceptual qualities of sounds. In this work, we use an artificial immune system (AIS) called opt-aiNet to search for combinations of musical instrument sounds that minimize the distance to a target sound encoded in a fitness function. Opt-aiNet is capable of finding multiple solutions in parallel while preserving diversity, proposing alternative orchestrations for the same target sound that are different among themselves. We performed a listening test to evaluate the subjective similarity and diversity of the orchestrations.

I Keep Counting: An Experiment in Human/AI Co-creative Songwriting

“…Even if the team was almost the only one in the contest to generate structure from a (very simple) AI method (see Section 3.1, and Huang et al (2020, Figure 1)), many decisions on the arrangement were human-made, specifically which instrumentss and accompaniment patterns enter and when. Although new services helping composers with these tasks are mainly black boxes, automatic arrangement and orchestration is an active research field (Abreu et al, 2016;Crestel and Esling, 2017;Tsushima et al, 2018) that will bring new possibilities in the next years.…”

Imitative Computer-Aided Musical Orchestration with Biologically Inspired Algorithms

“…Carpentier et al [20,21,76,18,22] adopted a bottom-up approach that relies on timbre similarity and evolutionary computation to search for instrument combinations that approximate the reference. The bottom-up approach represents a paradigm shift toward generative CAMO [21,18,33,1,15], where the timbre of instrument combinations is compared with the timbre of the reference sound via descriptors of timbre. Currently, there are two generative CAMO frameworks, the Orch* family of CAMO systems based on GA [18,22,33,26], and CAMO-AIS [1,15], which uses an artificial immune system (AIS).…”

“…The bottom-up approach represents a paradigm shift toward generative CAMO [21,18,33,1,15], where the timbre of instrument combinations is compared with the timbre of the reference sound via descriptors of timbre. Currently, there are two generative CAMO frameworks, the Orch* family of CAMO systems based on GA [18,22,33,26], and CAMO-AIS [1,15], which uses an artificial immune system (AIS). Orch* comprises three CAMO systems, namely Orchidée [20,21,76,18,22], Orchids [33,32], and Orchidea [26].…”

“…However, in practice, the orchestrations returned by Orchidée, for example, were all very similar to one another, commonly differing by only one musical instrument sound [15]. Caetano et al [1,15] proposed to use an AIS called opt-aiNet to search for combinations of musical instrument sounds that minimize the distance to a reference sound encoded in a fitness function. CAMO-AIS relies on single-objective optimization (SOO) and the multi-modal ability of opt-aiNet to find multiple solutions in parallel.…”

“…Therefore, it would be impractical and rather confusing to try to exhaustively describe all of them. Instead, this section will focus on CAMO-AIS [1,15], which closely follows the framework proposed by Carpentier et al [20,21,17,18,22,19]. Section 4 will explore the main differences between CAMO-AIS and Orchidée, especially the differences between the optimization method adopted by each and the consequence in terms of diversity of orchestrations.…”