Expressive Concatenative Synthesis by Reusing Samples from Real Performance Recordings

Maestre, Esteban; Ramírez, Rafael; Kersten, Stefan; Serra, Xavier

doi:10.1162/comj.2009.33.4.23

Cited by 21 publications

(16 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…1 Note that the reconstructed audio examples sound rather unnatural because the experiment is not conducted in a full concatenative synthesis framework. In particular we use a uniform grain duration of 100ms and impose no temporal constraints, whereas a full concatenative synthesis system typically segments sounds using detected onsets and includes temporal constraints for continuity, and therefore is able to synthesise much more natural attack/sustain dynamics [Maestre et al, 2009]. Our method shows promise as the timbral component of a multi-attribute search which could potentially be used in concatenative synthesis, as well as other applications requiring timbral search from audio examples (e.g.…”

Section: Resultsmentioning

confidence: 99%

“…Concatenative synthesisers typically operate not only on timbre, but use pitch and duration as well as temporal continuity constraints in their search strategy, and then modify the selected grains further to improve the match [Maestre et al, 2009]. While recognising the importance of these aspects in a full concatenative synthesis system, we designed an experiment in which the role of pitch, duration and temporal continuity were minimised, by excluding such factors from grain construction/analysis/resynthesis, and also by selecting audio excerpts whose variation is primarily timbral.…”

Section: Experiments 2: Concatenative Synthesismentioning

confidence: 99%

See 1 more Smart Citation

Learning Timbre Analogies from Unlabelled Data by Multivariate Tree Regression

Stowell

Plumbley

2011

Journal of New Music Research

View full text Add to dashboard Cite

Applications such as concatenative synthesis (audio mosaicing) and query-by-example require the ability to search a database using a sound which is qualitatively different from the actual desired result -for example when using vocal queries to retrieve nonvocal sound. Standard query techniques such as nearest neighbours do not account for this difference between source and target; they perform retrieval but do not learn to make timbral analogies. This paper addresses this issue by considering timbral query as a multivariate regression problem from one timbre distribution onto another. We develop a novel variant of multivariate tree regression:given only a set of unlabelled and unpaired samples from two distributions on the same space, the regression learns a cross-associative mapping which assumes general similarities in structure of the two distributions, yet can accommodate differences in shape at various scales. We demonstrate the technique with a synthetic example and with a concatenative synthesiser.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Experiments 2: Concatenative Synthesismentioning

confidence: 99%

Learning Timbre Analogies from Unlabelled Data by Multivariate Tree Regression

Stowell

Plumbley

2011

Journal of New Music Research

View full text Add to dashboard Cite

show abstract

“…Given the set of source distributions used for obtaining the mixed distribution (see Section V-A), the cost is computed as a weighted sum of negative log-likelihoods of the vector , each one computed given the corresponding th original distribution. This is expressed as (38) where represents the weight applied to each of the likelihoods, and is an additional cost related to the bow displacement of the current execution candidate. The value of used for weighting each th likelihood is set to the Mahalanobis distance from the target performance context point to the centroid of the th source distribution (see Section V-A), computed as in (25).…”

Section: B Cost Computationmentioning

confidence: 99%

“…1) Sample Retrieval: As a first step, a sample candidate list is generated for each th note in the input score by making use of different sample annotations: matching articulation, bow direction, played string, bow context, and silence context (see Section III-A). Then, sample retrieval is performed through an optimal path search in a similar fashion as presented in [38] after [39] (again making use of dynamic programming). In this work, the main novelty is the introduction of a measure of distance between target bowing control parameters and sample bowing control parameters, therefore enabling the search for samples by accounting for bowing parameter contours.…”

Section: Sample-based Sound Synthesismentioning

confidence: 99%

Statistical Modeling of Bowing Control Applied to Violin Sound Synthesis

Maestre

Blaauw

Bonada

et al. 2010

IEEE Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

Abstract-Excitation-continuous music instrument control patterns are often not explicitly represented in current sound synthesis techniques when applied to automatic performance. Both physical model-based and sample-based synthesis paradigms would benefit from a flexible and accurate instrument control model, enabling the improvement of naturalness and realism. We present a framework for modeling bowing control parameters in violin performance. Nearly non-intrusive sensing techniques allow for accurate acquisition of relevant timbre-related bowing control parameter signals. We model the temporal contour of bow velocity, bow pressing force, and bow-bridge distance as sequences of short Bézier cubic curve segments. Considering different articulations, dynamics, and performance contexts, a number of note classes are defined. Contours of bowing parameters in a performance database are analyzed at note-level by following a predefined grammar that dictates characteristics of curve segment sequences for each of the classes in consideration. As a result, contour analysis of bowing parameters of each note yields an optimal representation vector that is sufficient for reconstructing original contours with significant fidelity. From the resulting representation vectors, we construct a statistical model based on Gaussian mixtures suitable for both the analysis and synthesis of bowing parameter contours. By using the estimated models, synthetic contours can be generated through a bow planning algorithm able to reproduce possible constraints caused by the finite length of the bow. Rendered contours are successfully used in two preliminary synthesis frameworks: digital waveguide-based bowed string physical modeling and sample-based spectral-domain synthesis.

show abstract

“…More recently, Maestre et al used concatenative synthesis techniques aided by an expressive performance model to "generate an expressive audio sequence from the analysis of an arbitrary input score" [29]. Inspired by previous work by Schwarz, the authors gleaned segment information not only from analysis of the signal of an instrument note, but from the context of the score during the time that the note was played.…”

Section: Concatenative Synthesismentioning

confidence: 99%

Concatenative Synthesis for Novel Timbral Creation

Bilous¹

View full text Add to dashboard Cite

Concatenative Synthesis for Novel Timbral Creation James BilousModern day musicians rely on a variety of instruments for musical expression. Tones produced from electronic instruments have become almost as commonplace as those produced by traditional ones as evidenced by the plethora of artists who can be found composing and performing with nothing more than a personal computer. This desire to embrace technical innovation as a means to augment performance art has created a budding field in computer science that explores the creation and manipulation of sound for artistic purposes. One facet of this new frontier concerns timbral creation, or the development of new sounds with unique characteristics that can be wielded by the musician as a virtual instrument.This thesis presents Timcat, a software system that can be used to create novel timbres from prerecorded audio. Various techniques for timbral feature extraction from short audio clips, or grains, are evaluated for use in timbral feature spaces. Clustering is performed on feature vectors in these spaces and groupings are recombined using concatenative synthesis techniques in order to form new instrument patches.The results reveal that interesting timbres can be created using features extracted by both newly developed and existing signal analysis techniques, many common in other fields though not often applied to music audio signals. Several of the features employed also show high accuracy for instrument separation in randomly mixed tracks. Survey results demonstrate positive feedback concerning the timbres created by Timcat from electronic music composers, musicians, and music lovers alike.iv ACKNOWLEDGMENTS Thank you to my family, friends, and advisers who have provided strength, kindness and guidance when it was needed most.

show abstract

Expressive Concatenative Synthesis by Reusing Samples from Real Performance Recordings

Cited by 21 publications

References 26 publications

Learning Timbre Analogies from Unlabelled Data by Multivariate Tree Regression

Learning Timbre Analogies from Unlabelled Data by Multivariate Tree Regression

Statistical Modeling of Bowing Control Applied to Violin Sound Synthesis

Concatenative Synthesis for Novel Timbral Creation

Contact Info

Product

Resources

About