Kernel Machines Beat Deep Neural Networks on Mask-Based Single-Channel Speech Enhancement

Abstract

We apply a fast kernel method for mask-based single-channel speech enhancement. Specifically, our method solves a kernel regression problem associated to a non-smooth kernel function (exponential power kernel) with a highly efficient iterative method (EigenPro). Due to the simplicity of this method, its hyper-parameters such as kernel bandwidth can be automatically and efficiently selected using line search with subsamples of training data. We observe an empirical correlation between the regression loss (mean square error) and regular metrics for speech enhancement. This observation justifies our training target and motivates us to achieve lower regression loss by training separate kernel model per frequency subband. We compare our method with the state-of-the-art deep neural networks on mask-based HINT and TIMIT. Experimental results show that our kernel method consistently outperforms deep neural networks while requiring less training time.Index Termslarge-scale kernel machines, deep neural networks, speech enhancement, exponential power kernel, automatic hyper-parameter selection

Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHS

“…If it is restricted to the unit sphere, the RKHS of the exponential power kernel with γ < 1 is even larger than that of NTK. This result partially explains the observation in [19] that the best performance is attained by a highly non-smooth exponential power kernel with γ < 1. Geifman et al [18] applied the exponential power kernel and the NTK to classification and regression tasks on the UCI dataset and other large scale datasets.…”

“…For example, Belkin et al [7] showed experimentally that the Laplace kernel and neural networks had similar performance in fitting random labels. In the task of speech enhancement, exponential power kernels K γ,σ exp (x, y) = e − x−y γ /σ , which include the Laplace kernel as a special case, outperform deep neural networks with even shorter training time [19]. The experiments in [18] also exhibited similar performance of the Laplace kernel and the NTK.…”

“…Exponential Power Kernel The exponential power kernel [19] with γ > 0 and σ > 0 is given by K γ,σ exp (x, y) = exp − x−y γ σ . If x and y are restricted to the sphere S d−1 , we have…”

Layer-Wise Relevance Propagation Based Sample Condensation for Kernel Machines

“…Instead, we use them to study the ability of our approach to sample condensation without impairing the classification performance of kernel machines. The power of kernel machines themselves as classifier has already been demonstrated in the literature [4,9].…”

“…Compared to deep neural networks (DNN), they can be interpreted as two-layer NNs. Despite the simplicity, however, kernel machines turned out to be a good alternative to DNNs, capable of matching and even surpassing their performance while utilizing less computational resources in training [8,9].…”