VPNET: Variable Projection Networks

Abstract

In this paper, we introduce VPNet, a novel model-driven neural network architecture based on variable projection (VP). Applying VP operators to neural networks results in learnable features, interpretable parameters, and compact network structures. This paper discusses the motivation and mathematical background of VPNet and presents experiments. The VPNet approach was evaluated in the context of signal processing, where we classified a synthetic dataset and real electrocardiogram (ECG) signals. Compared to fully connected and one-dimensional convolutional networks, VPNet offers fast learning ability and good accuracy at a low computational cost of both training and inference. Based on these advantages and the promising results obtained, we anticipate a profound impact on the broader field of signal processing, in particular on classification, regression and clustering problems.

Road Abnormality Detection Using Piezoresistive Force Sensors and Adaptive Signal Models

“…In this setup, if we consider a constant vehicle velocity of 50 km/h, a single tire revolution would be represented by 143 data points. This is a sufficient amount of data for modeling the sensor output using the representation described in section V. In fact, similar signal models were used in [32] to represent the so-called QRS-complexes of ECG recordings. These signals consisted of 100 data points and shared morphological similarities with the wheel sensor measurements.…”

“…In this section we describe the application of VP-NET to the road abnormality detection problem. VP-NET is a special neural-network architecture introduced in [32] containing socalled variable projection layers. These layers (henceforth referred to as VP-layers) are capable of solving (6) and passing the results to a conventional fully-connected neural network.…”

“…In contrast, the best performing convolutional neural network solving the abnormal surface detection problem used a kernel size of 25 increasing the complexity of the network architecture, and thus the computational cost of the training process. Additionally, the parameters of the VP-layer are interpretable representing the location and the width of the spike-like waveforms in the input [32], whereas the weights of the convolutional layers have no such physical meaning.…”

Arrhythmia Detection Using Spiking Variable Projection Neural Networks

“…The functions φ j (•; θ)'s are chosen a priori to the problem in question in such a way that the parameter θ usually represents physical quantities, e.g., attenuation coefficients, calibration parameters, dominant frequencies, etc. Therefore, VPs provide an interpretable representation of the input x, which motivated the construction of VPNNs [2].…”

“…The VP layer has two modes of operations, its output could be either the projected signal x = Φ(θ)c or the coefficients of the projection c = Φ + (θ)x. Note that both cases are differentiable [1], thus backpropagation can be used for training [2]. In this work, all the VP layers forward the coefficient vector c of the orthogonal projection of the input x with respect to a set of parameterized Hermite functions φ(•; θ) where θ = (τ, λ).…”