Neural Melody Composition from Lyrics

Bao, Hangbo; Huang, Shaohan; Wei, Furu; Cui, Lei; Wu, Yu; Tan, Chuanqi; Piao, Songhao; Zhou, Ming

doi:10.1007/978-3-030-32233-5_39

Cited by 18 publications

(16 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A recently proposed ALYSIA songwriting system [8] is a lyrics-conditioned melody generation system based on exploiting a random forest model, which can predict the pitch and rhythm of notes to determine the accompaniments for lyrics. When given Chinese lyrics, melody and exact alignment are predicted in a lyrics-conditional melody composition framework [9], which is an end-to-end neural network model including RNN-based lyrics encoder, RNNbased context melody encoder, and a hierarchical RNN decoder. The authors create large-scale Chinese language lyricsmelody dataset to evaluate the proposed learning model.…”

Section: Related Workmentioning

confidence: 99%

“…Generating a melody from lyrics is to predict a melodic sequence when given lyrics as a condition. Existing works, e.g., Markov models [7], random forests [8], and recurrent neural network (RNN) [9], can generate lyrics-conditioned music melody. However, these methods cannot ensure that the distribution of generated data is consistent with that of real samples.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Conditional LSTM-GAN for Melody Generation from Lyrics

Srivastava

Canales

2021

ACM Trans. Multimedia Comput. Commun. Appl.

View full text Add to dashboard Cite

Melody generation from lyrics has been a challenging research issue in the field of artificial intelligence and music, which enables us to learn and discover latent relationships between interesting lyrics and accompanying melodies. Unfortunately, the limited availability of a paired lyrics–melody dataset with alignment information has hindered the research progress. To address this problem, we create a large dataset consisting of 12,197 MIDI songs each with paired lyrics and melody alignment through leveraging different music sources where alignment relationship between syllables and music attributes is extracted. Most importantly, we propose a novel deep generative model, conditional Long Short-Term Memory (LSTM)–Generative Adversarial Network for melody generation from lyrics, which contains a deep LSTM generator and a deep LSTM discriminator both conditioned on lyrics. In particular, lyrics-conditioned melody and alignment relationship between syllables of given lyrics and notes of predicted melody are generated simultaneously. Extensive experimental results have proved the effectiveness of our proposed lyrics-to-melody generative model, where plausible and tuneful sequences can be inferred from lyrics.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Conditional LSTM-GAN for Melody Generation from Lyrics

Srivastava

Canales

2021

ACM Trans. Multimedia Comput. Commun. Appl.

View full text Add to dashboard Cite

show abstract

“…Xiaobing Li, composer, Professor of the Central Conservatory of Music (CCOM) 2 in China, and the head 3 of the Department of Music AI and Information Technology 4 in CCOM, Director of the Art and Artificial Intelligence Committee of the Chinese Association for Artificial Intelligence, and chief expert of major national social science projects. He is a researcher and advocator of "3D music", and an expert in electronic music, computer music, and music technology.…”

Section: Xiaobing LImentioning

confidence: 99%

A Tutorial on AI Music Composition

Tan

2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

AI music composition is one of the most attractive and important topics in artificial intelligence, music, and multimedia. The typical tasks in AI music composition include melody generation, song writing, accompaniment generation, arrangement, performance generation, timbre rendering, sound generation, and singing voice synthesis, which cover different modalities (e.g., symbolic music score, sound) and well match to the theme of ACM Multimedia. As the rapid development of artificial intelligence techniques such as content creation and deep learning, AI based music composition has achieved rapid progress, but still encountered a lot of challenges. A thorough introduction and review on the basics, the research progress, as well as how to address the challenges in AI music composition are timely and necessary for a broad audience working on artificial intelligence, music, and multimedia. In this tutorial, we will first introduce the background of AI music composition, including music basics and deep learning techniques for music composition. Then we will introduce AI music composition from two perspectives: 1) key components, which include music score generation, music performance generation, and music sound generation; 2) advanced topics, which include music structure/form/style/emotion modeling, timbre synthesis/transfer/mixing, etc. At last, we will point out some research challenges and future directions in AI music composition. This tutorial can serve both academic researchers and industry practitioners working on AI music composition.

show abstract

“…Moreover, little attention has been paid to Chinese music generation with deep learning techniques, especially for modeling the music style of Chinese music, though some researchers utilize Seq2Seq model to create multi-track Chinese popular songs from scratch [34] or generate melody of Chinese popular songs with given lyrics [1]. The existing generation algorithms for Chinese traditional songs are mostly based on non-deep models such as Markov models [13], genetic algorithms [33].…”

Section: Related Workmentioning

confidence: 99%

MG-VAE: Deep Chinese Folk Songs Generation with Specific Regional Styles

Luo

Yang

et al. 2019

Lecture Notes in Electrical Engineering

View full text Add to dashboard Cite

Regional style in Chinese folk songs is a rich treasure that can be used for ethnic music creation and folk culture research. In this paper, we propose MG-VAE, a music generative model based on VAE (Variational Auto-Encoder) that is capable of capturing specific music style and generating novel tunes for Chinese folk songs (Min Ge) in a manipulatable way. Specifically, we disentangle the latent space of VAE into four parts in an adversarial training way to control the information of pitch and rhythm sequence, as well as of music style and content. In detail, two classifiers are used to separate style and content latent space, and temporal supervision is utilized to disentangle the pitch and rhythm sequence. The experimental results show that the disentanglement is successful and our model is able to create novel folk songs with controllable regional styles. To our best knowledge, this is the first study on applying deep generative model and adversarial training for Chinese music generation. 2 Related Work RNN (Recurrent Neural Network) is one of the most earliest models introduced into the domain of deep music generation. Researchers employ RNNs to model the music structure and generate different formats of music, including monophonic folk melodies [30], rhythm composition [23], expressive music performance [27], multi-part music harmonization [31]. Other recent studies have

show abstract

Neural Melody Composition from Lyrics

Cited by 18 publications

References 12 publications

Conditional LSTM-GAN for Melody Generation from Lyrics

Conditional LSTM-GAN for Melody Generation from Lyrics

A Tutorial on AI Music Composition

MG-VAE: Deep Chinese Folk Songs Generation with Specific Regional Styles

Contact Info

Product

Resources

About