CycleGAN-Based Singing/Humming to Instrument Conversion Technique

Abstract

In this research, singing/humming to instrument conversion techniques are proposed. In humming to instrument, two models based on cycle-consistent adversarial networks (CycleGAN) on viola are experimented. From the objective and subjective evaluations conducted, the converted audio is more similar to viola compared to humming, and the quality of the converted sound is fair to listeners. In singing to instrument, to fix the problem of the gap between singing and instrument, a dual conversion model consisting of singing to humming and humming to instrument is proposed. The objective and subjective experimental results show that the dual conversion has better converted audio quality than conversion by singing to instrument directly.