Some theoretical results of learning theory based on random sets in set‐valued probability space

Abstract

PurposeThe purpose of this paper is to introduce some basic knowledge of statistical learning theory (SLT) based on random set samples in set‐valued probability space for the first time and generalize the key theorem and bounds on the rate of uniform convergence of learning theory in Vapnik, to the key theorem and bounds on the rate of uniform convergence for random sets in set‐valued probability space. SLT based on random samples formed in probability space is considered, at present, as one of the fundamental theories about small samples statistical learning. It has become a novel and important field of machine learning, along with other concepts and architectures such as neural networks. However, the theory hardly handles statistical learning problems for samples that involve random set samples.Design/methodology/approachBeing motivated by some applications, in this paper a SLT is developed based on random set samples. First, a certain law of large numbers for random sets is proved. Second, the definitions of the distribution function and the expectation of random sets are introduced, and the concepts of the expected risk functional and the empirical risk functional are discussed. A notion of the strict consistency of the principle of empirical risk minimization is presented.FindingsThe paper formulates and proves the key theorem and presents the bounds on the rate of uniform convergence of learning theory based on random sets in set‐valued probability space, which become cornerstones of the theoretical fundamentals of the SLT for random set samples.Originality/valueThe paper provides a studied analysis of some theoretical results of learning theory.

Bounds on the rate of convergence of learning processes based on random sets and set‐valued probability

“…It is easily seen that when the set-valued probability in this theorem is a single point set we arrive at Theorem 14 in Ha et al (2009).…”

“…(Ha et al, 2009) Let (V,A,p) be a set-valued probability space, f n (n $ 1) be compact convex random set sequences from (V,A,p) to: R; a n ¼ inff n ; b n ¼ supf n ; A 1 ¼ inf{a n } # a n # sup{a n } ¼ A 2 ;…”

“…In this paper, utilizing the partial relation " # " and the properties of set-valued probability we revisited the nonconstructive distributiondependent bounds on the rate of uniform convergence of learning processes, which is more broader than those in Ha et al (2009). Although the key theorem supplies the sufficient and necessary conditions of the empirical risk minimization (ERM) principle based on random sets and set-valued probability, it does not tell us what learning methods can satisfy these conditions.…”

Support Vector Machine Based on Random Set Samples

“…Therefore, more and more scholars are engaged in the research on statistical learning theory based on non-real valued random samples. [11][12][13] In aspect of the theoretical foundations, Ha et al 5 initially built uncertainty statistical learning theory in 2010, and provided the statistical learning theory based on non-real valued random samples on non-probability measure space. In aspect of SVM, some scholars have achieved certain developments.…”

3The Structural Risk Minimization principle on set-valued probability space

“…Based on random sets we discussed the empirical risk minimization (ERM) principle, the key theorem [3,4] and the bounds on the rate of uniform convergence of the learning [5]. The bounds on the rate of uniform processes convergence of learning processes include two portion of the empirical risk of training samples and confidence interval.…”