Kernel Risk-Sensitive Loss: Definition, Properties and Application to Robust Adaptive Filtering

Chen, Badong; Liu, Xing; Xu, Bin; Zhao, Haiquan; Zheng, Nanning; Prı́ncipe, José C.

doi:10.1109/tsp.2017.2669903

Cited by 146 publications

(76 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The MCC as a measure of the information content in information theoretic learning was developed by principe with his team to deal with error distributions with non-Gaussian characteristics, and it has been widely used in pattern classification, feature selection, dimension reduction and adaptive filtering [21][22][23][24][25]. The MCC indicates the similarity between the predicted output and the real sample in the correntropy sense; it shows good robustness for nonlinear and non-Gaussian data processing, such as determining whether electricity is suitable for the prediction of time series non-stationary and time-varying predictions [26].…”

Section: Maximum Correntropy Criterionmentioning

confidence: 99%

Electricity Consumption Forecasting Scheme via Improved LSSVM with Maximum Correntropy Criterion

Duan

Qiu

et al. 2018

Entropy

View full text Add to dashboard Cite

Abstract:In recent years, with the deepening of China's electricity sales side reform and electricity market opening up gradually, the forecasting of electricity consumption (FoEC) becomes an extremely important technique for the electricity market. At present, how to forecast the electricity accurately and make an evaluation of results scientifically are still key research topics. In this paper, we propose a novel prediction scheme based on the least-square support vector machine (LSSVM) model with a maximum correntropy criterion (MCC) to forecast the electricity consumption (EC). Firstly, the electricity characteristics of various industries are analyzed to determine the factors that mainly affect the changes in electricity, such as the gross domestic product (GDP), temperature, and so on. Secondly, according to the statistics of the status quo of the small sample data, the LSSVM model is employed as the prediction model. In order to optimize the parameters of the LSSVM model, we further use the local similarity function MCC as the evaluation criterion. Thirdly, we employ the K-fold cross-validation and grid searching methods to improve the learning ability. In the experiments, we have used the EC data of Shaanxi Province in China to evaluate the proposed prediction scheme, and the results show that the proposed prediction scheme outperforms the method based on the traditional LSSVM model.

show abstract

Section: Maximum Correntropy Criterionmentioning

confidence: 99%

Electricity Consumption Forecasting Scheme via Improved LSSVM with Maximum Correntropy Criterion

Duan

Qiu

et al. 2018

Entropy

View full text Add to dashboard Cite

show abstract

“…where J 1 , J 2 have been shown in Equation 8. The optimal decision function under our ensemble loss function is obtained by setting the derivative of J to zero, 5 shows an example of two convex functions J 1 and J 2 with the optimal point f * 1 and f * 2 respectively.…”

Section: Bayes Consistencymentioning

confidence: 99%

relf: robust regression extended with ensemble loss function

2018

View full text Add to dashboard Cite

Ensemble techniques are powerful approaches that combine several weak learners to build a stronger one. As a meta-learning framework, ensemble techniques can easily be applied to many machine learning methods. Inspired by ensemble techniques, in this paper we propose an ensemble loss functions applied to a simple regressor. We then propose a half-quadratic learning algorithm in order to find the parameter of the regressor and the optimal weights associated with each loss function. Moreover, we show that our proposed loss function is robust in noisy environments. For a particular class of loss functions, we show that our proposed ensemble loss function is Bayes consistent and robust. Experimental evaluations on several data sets demonstrate that the our proposed ensemble loss function significantly improves the performance of a simple regressor in comparison with state-of-the-art methods.Loss functions are fundamental components of machine learning systems and are used to train the parameters of the learner model. Since standard training methods aim to determine the parameters that minimize the average value of the loss given an annotated training set, loss functions are crucial for successful trainings [49,55]. Bayesian estimators are obtained by minimizing the expected loss function. Different loss functions lead to different Optimum Bayes with possibly different characteristics. Thus, in each environment the choice of the underlying loss function is important, as it will impact the performance [44,48].Lettingθ denote the estimated parameter of a correct parameter θ, the loss function L(θ, θ) is a positive function which assigns a loss value to each estimation, indicating how inaccurate the estimation is [42]. Loss functions assign a value to each sample, indicating how much that sample contributes to solving the optimization problem. Each loss function comes with its own advantages and disadvantages. In order to put our results in context, we start by reviewing three popular loss functions (0-1, Ramp and Sigmoid) and we will give an overview of their advantages and disadvantages.Loss functions assign a value to each sample representing how much that sample contributes to solving the optimization problem. If an outlier is given a very large value by the loss function, it might dramatically affect the decision function [18]. The 0-1 loss function is known as a robust loss because it assigns value 1 to all misclassified samples -including outliers -and thus an outlier does not influence the decision function, leading to a robust learner. On the other hand, the 0-1 loss penalizes all misclassified samples equally with value 1, and since it does not enhance the margin, it cannot be an appropriate choice for applications with margin importance [49].The Ramp loss function, as another type of loss functions, is defined similarly to the 0-1 loss function with the only difference that ramp loss functions also penalize some correct samples, those with small margins. This minor difference makes the Ramp loss function appro...

show abstract

“…On the other hand, when kernel bandwidth becomes larger, the robustness will be significantly reduced when outliers appear. To achieve better performance, a new similarity measure, i.e., KRSL, was proposed in [37], which can be more "convex" and thus can achieve faster convergence rate and higher filtering accuracy while maintaining the robustness to outliers.…”

Section: The Kernel Risk-sensitive Loss (Krsl) Algorithmmentioning

confidence: 99%

“…However, correntropic loss (C-Loss) is a non-convex function, and may converge slowly, especially when the initial value is far away from the optimal value. Recently, a modified similarity measure called kernel risk-sensitive loss (KRSL) was derived in [37], which can be more "convex" and can achieve faster convergence speed and higher filtering accuracy, while maintaining the robustness to outliers. After applying KRSL to adaptive filtering, a new robust KAF algorithm was accordingly proposed [37].…”

Section: Introductionmentioning

confidence: 99%

“…Recently, a modified similarity measure called kernel risk-sensitive loss (KRSL) was derived in [37], which can be more "convex" and can achieve faster convergence speed and higher filtering accuracy, while maintaining the robustness to outliers. After applying KRSL to adaptive filtering, a new robust KAF algorithm was accordingly proposed [37]. It is called the minimum kernel risk-sensitive loss (MKRSL) algorithm, which could be superior to some existing methods and is attracting growing attention.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Quantized Kernel Learning Algorithm Using a Minimum Kernel Risk-Sensitive Loss Criterion and Bilateral Gradient Technique

Luo

Deng

Wang

et al. 2017

Entropy

View full text Add to dashboard Cite

Abstract:Recently, inspired by correntropy, kernel risk-sensitive loss (KRSL) has emerged as a novel nonlinear similarity measure defined in kernel space, which achieves a better computing performance. After applying the KRSL to adaptive filtering, the corresponding minimum kernel risk-sensitive loss (MKRSL) algorithm has been developed accordingly. However, MKRSL as a traditional kernel adaptive filter (KAF) method, generates a growing radial basis functional (RBF) network. In response to that limitation, through the use of online vector quantization (VQ) technique, this article proposes a novel KAF algorithm, named quantized MKRSL (QMKRSL) to curb the growth of the RBF network structure. Compared with other quantized methods, e.g., quantized kernel least mean square (QKLMS) and quantized kernel maximum correntropy (QKMC), the efficient performance surface makes QMKRSL converge faster and filter more accurately, while maintaining the robustness to outliers. Moreover, considering that QMKRSL using traditional gradient descent method may fail to make full use of the hidden information between the input and output spaces, we also propose an intensified QMKRSL using a bilateral gradient technique named QMKRSL_BG, in an effort to further improve filtering accuracy. Short-term chaotic time-series prediction experiments are conducted to demonstrate the satisfactory performance of our algorithms.

show abstract

Kernel Risk-Sensitive Loss: Definition, Properties and Application to Robust Adaptive Filtering

Cited by 146 publications

References 38 publications

Electricity Consumption Forecasting Scheme via Improved LSSVM with Maximum Correntropy Criterion

Electricity Consumption Forecasting Scheme via Improved LSSVM with Maximum Correntropy Criterion

relf: robust regression extended with ensemble loss function

A Quantized Kernel Learning Algorithm Using a Minimum Kernel Risk-Sensitive Loss Criterion and Bilateral Gradient Technique

Contact Info

Product

Resources

About