Vector quantization based on ε-insensitive mixture models

Abstract

a b s t r a c tLaplacian mixture models have been used to deal with heavy-tailed distributions in data modeling problems. We consider an extension of Laplacian mixture models, which consists of ε-insensitive component distributions. An EM-type learning algorithm is derived for the maximum likelihood estimation of the proposed mixture model. The E-step is formulated in the usual way, while the M-step is formulated as the dual optimization problem instead of the primal optimization problem.Additionally, the convergence proof for ε¼0 is accomplished. As an analogy to the k-means algorithm, we obtain what we call the ei-means algorithm in a certain limit of the learning algorithm. The derived algorithm is applied to approximate computation of rate-distortion functions associated with the ε-insensitive loss function. Then, it is demonstrated by synthetic data and real-world Spambase data that with appropriate selection of the ε value, the model is able to tolerate small percentage of noisy data.

Generalized Dirichlet-Process-Means for Robust and Maximum Distortion Criteria

“…The flow of the proof is almost the same as the proof of Theorem 1. We show that the objective function (4) monotonically decreases when the k-th cluster center θ k is newly updated toθ k by (24). More specifically, we prove that,L f (θ k ) ≥L f (θ k ), whereL f (θ k ) is the sum of the terms related to θ k in (4), that is,L…”

“…That is, the algorithm converges in finite iterations for threshold δ. The proof of Theorem 1 is a generalization of the monotonic decreasing property of ei-means (ε = 0) proposed in [24], which corresponds to the case where f (z) = √ z and d φ (x, θ) = x − θ 2 . The proof of Theorem 1 is also interpreted by the Majorization-Minimization algorithm [25].…”

“…That is, the algorithm converges in finite iterations for threshold δ. The proof of Theorem 1 is a generalization of the monotonic decreasing property of ei-means (ε = 0) proposed in [24], which corresponds to the case where…”