One deep music representation to rule them all? A comparative analysis of different representation learning strategies

Abstract

Inspired by the success of deploying deep learning in the fields of Computer Vision and Natural Language Processing, this learning paradigm has also found its way into the field of Music Information Retrieval. In order to benefit from deep learning in an effective, but also efficient manner, deep transfer learning has become a common approach. In this approach, it is possible to reuse the output of a pre-trained neural network as the basis for a new learning task. The underlying hypothesis is that if the initial and new learning tasks show commonalities and are applied to the same type of input data (e.g. music audio), the generated deep representation of the data is also informative for the new task. Since, however, most of the networks used to generate deep representations are trained using a single initial learning source, their representation is unlikely to be informative for all possible future tasks. In this paper, we present the results of our investigation of what are the most important factors to generate deep representations for the data and learning tasks in the music domain. We conducted this investigation via an extensive empirical study that involves multiple learning sources, as well as multiple deep learning architectures with varying levels of information sharing between sources, in order to learn music representations. We then validate these representations considering multiple target datasets for evaluation. The results of our experiments yield several insights on how to approach the design of methods for learning widely deployable deep data representations in the music domain.

Randomly Weighted CNNs for (Music) Audio Classification

“…Few related works exist in the audio field -and every randomly weighted neural network we found in the audio literature was a mere baseline [2,7,24]. Inspired by previous computer vision works, we study which audio architectures work the best via evaluating how nontrained CNNs perform as feature extractors.…”

Large Scale Audio-Visual Video Analytics Platform for Forensic Investigations of Terroristic Attacks

“…A similar approach is followed in Section 3. Recent attempts to learn audio embeddings and similarity functions with neural networks has shown promising results [7]. Video Analysis Video analytics software makes surveillance systems more efficient, by reducing the workload on security and management authorities.…”

Transfer Learning of Artist Group Factors to Musical Genre Classification

“…Considering that lower layers of DC-NNs usually capture lower-level features such as edges from images or spectrograms, we hypothesized that sharing lower layers among the various DCNNs can be effective under the scenario where multiple learning sources are available. With this approach, one can expect that it not only ensures sufficient specialization on taskspecific upper layers, but also benefits from regularization effects on lower layers [14]. Joint learning of multiple tasks with shared layers can prevent the shared layer to overfit for a specific task, instead learning underlying factors that have commonalities required across tasks [6,19].…”

“…To overcome these potential problems, we therefore apply a label pre-processing step, obtaining Artist Group Factors (AGF) as learning targets, rather than individual artist identities. Finally, we train Deep Convolutional Neural Networks (DCNNs) employing different learning setups, ranging from targeting genre and various types of AGFs with individual networks, to employing a shared architecture as introduced in multiple previous Multi-Task Learning (MTL) works [2,3,6,14,16,18,24,25].…”