A two-stage optimized robust kernel density estimation for Bayesian classification with outliers

Abstract

A Bayesian classifier based on kernel density estimation (KDE) is an effective routine for continuous data classification. However, the presence of outliers and inappropriate selection of bandwidth may cause inaccurate and distorted estimations of the density function. This paper presents an improved Bayesian classification approach by employing an optimized robust kernel density estimation (ORKDE). The impact of outliers can be down-weighted by a two-stage optimization in the conditional probability density calculation. In the first stage, an improved hybrid rice optimization (HRO) algorithm is proposed for selecting the bandwidth of a Gaussian KDE using a mixed cross-validation function. It incorporates Levy flight and elite reverse learning strategy to enhance the global optimization searching capability. In the second stage, a soft-redescending Welsch function is applied as the loss function to reduce the weights of outliers using the M-estimation in the kernel space. The relative efficiency index is used to derive the tuning parameter of the influence function. A kernelized iteratively re-weighted least-squares procedure employs both the tuning parameter and the quantile rule for effectively calculating the optimal weights of samples. With the proposed ORKDE method, the Naive Bayesian classifier enhances the generalization in the classification with outliers. The effectiveness of our method is demonstrated by the experiments using synthetic and real-world datasets.