Reverb Conversion Of Mixed Vocal Tracks Using An End-To-End Convolutional Deep Neural Network

Abstract

Reverb plays a critical role in music production, where it provides listeners with spatial realization, timbre, and texture of the music. Yet, it is challenging to reproduce the musical reverb of a reference music track even by skilled engineers. In response, we propose an end-to-end system capable of switching the musical reverb factor of two different mixed vocal tracks. This method enables us to apply the reverb of the reference track to the source track to which the effect is desired. Further, our model can perform de-reverberation when the reference track is used as a dry vocal source. The proposed model is trained in combination with an adversarial objective, which makes it possible to handle high-resolution audio samples. The perceptual evaluation confirmed that the proposed model can convert the reverb factor with the preferred rate of 64.8%. To the best of our knowledge, this is the first attempt to apply deep neural networks to converting music reverb of vocal tracks.

GibbsDDRM: A Partially Collapsed Gibbs Sampler for Solving Blind Inverse Problems with Denoising Diffusion Restoration

“…Comparison methods. We evaluate the proposed method against three baselines: Reverb Conversion (RC) (Koo et al, 2021), Music Enhancement (ME) (Kandpal et al, 2022), and Unsupervised Dereverberation (UD) (Saito et al, 2022). RC is a state-of-the-art, end-to- (Koo et al, 2021) 5.69 0.02 7.23 Music Enhancement (Kandpal et al, 2022) 7.51 −23.9 7.92 Unsupervised Dereverberation (Saito et al, 2022) 4 Evaluation metrics.…”

“…We evaluate the proposed method against three baselines: Reverb Conversion (RC) (Koo et al, 2021), Music Enhancement (ME) (Kandpal et al, 2022), and Unsupervised Dereverberation (UD) (Saito et al, 2022). RC is a state-of-the-art, end-to- (Koo et al, 2021) 5.69 0.02 7.23 Music Enhancement (Kandpal et al, 2022) 7.51 −23.9 7.92 Unsupervised Dereverberation (Saito et al, 2022) 4 Evaluation metrics. For quantitative comparison of the different methods, the metrics are the scale-invariant signalto-distortion ratio (SI-SDR) (Roux et al, 2019) improvement, the Fréchet Audio Distance (FAD) (Kilgour et al, 2018), and the speech-to-reverberation modulation energy ratio (SRMR) (Santos et al, 2014).…”

“…We use an exponential moving average over model parameters with a rate of 0.9999 (Song & Ermon, 2020). , 1112 ) from those of our test dataset (Koo et al, 2021). We input pairs of wet and dry signals since this method needs them for dereverberation.…”

End-to-end Music Remastering System Using Self-supervised and Adversarial Training

“…An end-to-end approach of converting black-box music effects with already processed audio tracks comes in need upon the loss of original dry source tracks or unavailability of replicating the setup of the mastering chain at the time, which may occur especially with old recordings. [8] introduced a system that interchanges the musical reverberant effects of two differently processed vocal tracks yet required massive storage of already-processed data to train. We propose an end-to-end remastering system that thoroughly converts the originally mastered effects to the desired style.…”

Visual Acoustic Matching

“…Blind estimation of the room impulse response from reverberant speech has also been explored [57,65]. In music production, acoustic matching is applied to change the reverberation to emulate that from a target space or processing algorithm [35,51]. Recent work conditions the target-audio generation on a low-dimensional audio embedding [59].…”