Towards democratizing music production with AI-Design of Variational Autoencoder-based Rhythm Generator as a DAW plugin

Tokui, Nao

doi:10.48550/arxiv.2004.01525

Cited by 1 publication

(3 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several added two more instruments (usually mapped to cymbals), and a few incorporated three more instruments (in general mapped to toms, cowbell, and percussion), adding up to a total of nine instruments. This number of instruments is the same found in previous research on drum sound classification (Herrera, Yeterian, & Gouyon, 2002), and used in implementations by Tokui (2020) and Gillick et al (2019). As a result, our chosen data representation for the encoding of one bar of 4/4 time comprises three vectors (for onsets, velocities, and microtimings) of size 864.…”

Section: Neural Network Architecturementioning

confidence: 80%

“…The application is still being tested and is not open to the public. Finally, M4L.RhythmVAE (Tokui, 2020) is a rhythm generation system that encodes onsets, velocities, and microtimings. It is based on GrooVAE but has a much simpler network architecture for faster training.…”

Section: Rhythm and Latent Spacesmentioning

confidence: 99%

“…. R-VAE is based on the Tensorflow.js VAE implementation tfjs-vae 10 and M4L.RhythmVAE (Tokui, 2020). While the former provides the Tensorflow backend for the VAE, the latter provides a data structure based on the one by Gillick et al (2019) that encodes the onsets of rhythms, their velocities, and microtimings.…”

Section: Neural Network Architecturementioning

confidence: 99%

See 2 more Smart Citations

Contemporary music genre rhythm generation with machine learning

Vigliensoni

McCallum

Maestre

et al. 2022

Journal of Creative Music Systems

View full text Add to dashboard Cite

In this article, we present research on customizing a variational autoencoder (VAE) neural network to learn models and play with musical rhythms encoded within a latent space. The system uses a data structure that is capable of encoding rhythms in simple and compound meter and can learn models from little training data. To facilitate the exploration of models, we implemented a visualizer that relies on the dynamic nature of the pulsing rhythmic patterns. To test our system in real-life musical practice, we collected small-scale datasets of contemporary music genre rhythms and trained models with them. We found that the non-linearities of the learned latent spaces coupled with tactile interfaces to interact with the models were very expressive and lead to unexpected places in composition and live performance musical settings. A music album was recorded and it was premiered at a major music festival using the VAE latent space on stage.

show abstract

Section: Neural Network Architecturementioning

confidence: 80%

Section: Rhythm and Latent Spacesmentioning

confidence: 99%