Basic filters for convolutional neural networks applied to music: Training or design?

Abstract

When convolutional neural networks are used to tackle learning problems based on music or other time series, raw one-dimensional data are commonly pre-processed to obtain spectrogram or mel-spectrogram coefficients, which are then used as input to the actual neural network. In this contribution, we investigate, both theoretically and experimentally, the influence of this pre-processing step on the network's performance and pose the question whether replacing it by applying adaptive or learned filters directly to the raw data can improve learning success. The theoretical results show that approximately reproducing mel-spectrogram coefficients by applying adaptive filters and subsequent time-averaging on the squared amplitudes is in principle possible. We also conducted extensive experimental work on the task of singing voice detection in music. The results of these experiments show that for classification based on Convolutional Neural Networks the features obtained from adaptive filter banks followed by time-averaging the squared modulus of the filters' output perform better than the canonical Fourier-transform-based mel-spectrogram coefficients. Alternative adaptive approaches with center frequencies or time-averaging lengths learned from training data perform equally well.

Quantum Harmonic Analysis on Lattices and Gabor Multipliers

“…Remark For Gabor multipliers c (ψ ⊗ ψ), Propositions 5.4 and 5.5 were proved in [14, Lem. 14], and have been used in the theory of convolutional neural networks [13].…”

On Orthogonal Projections for Dimension Reduction and Applications in Augmented Target Loss Functions for Learning Problems

“…Related principles of dimension reduction for other clinical classification problems in OCT have already been successfully applied in [9]. In the second experiment we aim to categorize musical instruments based on their spectrogram, see [18] for related results. Our utilized augmented target loss functions can increase the accuracy in both experiments.…”

A Frequency‐Uniform and Pitch‐Invariant Time‐Frequency Representation

“…The representation generalizes the first layer of the scattering transform [3]. Independently, a related transform was developed by Dörfler et al [4], which is equivalent to the mel spectrogram and therefore not frequencyuniform and pitch-invariant at the same time.…”