Deep Sigma Point Processes

Abstract

We introduce Deep Sigma Point Processes, a class of parametric models inspired by the compositional structure of Deep Gaussian Processes (DGPs). Deep Sigma Point Processes (DSPPs) retain many of the attractive features of (variational) DGPs, including mini-batch training and predictive uncertainty that is controlled by kernel basis functions. Importantly, since DSPPs admit a simple maximum likelihood inference procedure, the resulting predictive distributions are not degraded by any posterior approximations. In an extensive empirical comparison on univariate and multivariate regression tasks we find that the resulting predictive distributions are significantly better calibrated than those obtained with other probabilistic methods for scalable regression, including variational DGPsoften by as much as a nat per datapoint.

Uncertainty-aware Remaining Useful Life predictor

“…In this section, we briefly review the basic features of each method employed in our work. A more comprehensive analysis of the GP-based techniques used here can be found, for instance, in [6], from which the discussion below is largely inspired. In order to compare our GP-based methods with a strong Bayesian DL baseline, we additionally implement Monte Carlo Dropout (MCD) [38] and we apply it to the same RUL benchmark dataset.…”

“…Despite their many practical successes, variational inference methods often tend to provide overly confident uncertainty estimates [14,15]. A recent series of works [6,7] aimed to address this limitation by reformulating the variational inference scheme at the basis of SVGPs and DGPs. In particular, the authors note an inconsistency between the ELBO (the objective function to be optimized shown in Eq.…”

“…In [6], the DGP framework proposed in [5] is combined with the new loss function introduced in [7]. DSPPs arise from the necessity of overcoming one last theoretical obstacle: the direct application of the objective introduced in [7] to the DGP predictive distribution in Eq.…”

“…However, over the last 20 years, a number of efficient solutions to address the aforementioned limitations and to refine standard GP models have been proposed. In this work, we perform a thorough evaluation of three of such GP extensions: namely, Stochastic Variational Gaussian Processes (SVGPs) [2,3], Deep Gaussian Processes (DGPs) [4,5] and Deep Sigma Point Processes (DSPPs) [6,7] applied to prognostics. The evaluation is performed on a case study for RUL prediction on aircraft engines using the new Commercial Modular Aero-Propulsion System Simulation (N-CMAPSS) dataset from NASA.…”