Using ε-Nets for Linear Separation of Two Sets in a Euclidean Space R d

Abstract

This paper introduces the concept of e-separability. Necessary and sufficient conditions of e-separability are proved. It is proved that the problem of e-separability of two sets can be reduced to the trivial problem of separability of their disjoint e-nets. PROBLEM STATEMENT Let two finite setsA R d Ì and B R d Ì be given, and let their cardinalities be | | A n A = and | | B n B = . Assume that A B Ë conv and that B A Ë conv . In the simplest case when the convex envelopes of the sets A R d Ì and B R d Ì aredisjoint, they can be separated, i.e., a hyperplane can be found such that these sets will be located on the opposite sides of this hyperplane. Assume that the sets are inseparable, i.e., conv conv¹AE . The following question arises: when can these sets be separated by eliminating a small number of points from them, for example, e Î ( , ) 0 1 parts of their total number? At present, there exist many classification methods each of which has advantages and drawbacks. The Fischer discriminant analysis [1] is most popular. It is widely used in informatics branches such as machine learning, information search, and pattern recognition. The complexity of the algorithm of linear discriminant analysis is estimated to be O ndt t ( ) + 3 , where n is the number of observations in a training set, d is the number of features, and t n d = min( , ) [2]. Therefore, it is impossible to use the algorithm when the values of n and d are large.The Bayesian classifier [3] is optimal, it is easily implemented in software, and many classification methods are constructed on its basis. However, since, in practice, likelyhood functions of classes are reconstructed from finite data samples, the Bayesian classifier ceases to be optimal [4]. Its algorithmic complexity is estimated to be O nd ( ) [5]. A comparatively new support vector method well known in the literature as SVM [6] leads the maximization of the width of the separating band between classes owing to the optimal separating hyperplane principle. Thus, this method promotes a more reliable classification. However, at the same time, it is not resistant to noise in initial data. An essential drawback of the method is the absence of developed general methods for constructing straightening spaces and kernels that are most relevant to a concrete problem [7]. The complexity of the algorithm of the support vector method is estimated to be O n ( ) 3 [8]. The classification with the help of cluster analysis and also Wald's sequential analysis with the use of Kulbak's information measure are described in [9]. BASIC DEFINITIONS Definition 1. Sets A and B are called e-separable if there are A A 1 Ì and B B