Disjoint hard models for classification

Abstract

The paper describes a new approach for disjoint hard modelling of classes. This involves developing independent PC models for each group in the class, and calculating both the Q statistic (square prediction error) for each sample to the class model and a separate statistic about how well samples are classified within the projected PC space. The latter statistic can be applied to different types of classifiers, in this paper we choose to illustrate by Quadratic Discriminant Analysis (D statistic) and one class Support Vector Domain Description (SVDD) ( f-value). The two measures (Q and the classifier dependent statistic) are combined into a joint decision function which uniquely classifies each sample. The disjoint hard models are contrasted to conjoint models where PCA is performed on the entire dataset using both QDA and Support Vector Machines (SVMs) classifiers. The optimum number of PCs for each model is determined using the bootstrap, and model performance assessed on 100 test sets obtained using different iterative splits, using %PA (Predictive Ability) and %CR (Classification Rate). The method is illustrated using a dataset consisting of 293 samples from nine groups of polymers obtained using thermal profiling. The approach described, in this paper, has many of the advantages of one class disjoint models (e.g. SIMCA) and of conventional hard models, and is useful if it is known that all samples must belong to one of a series of known groups but each group has a very different structure.

One‐class classifiers

“…This does not prevent further groups being modelled, but the projection would be into the PC space of the original training set. The extension to hard one-class classifiers is conceivable [21]: in such circumstances, each class is modelled separately, but an unknown sample is unambiguously assigned to a single class, the one it most resembles. Sometimes SIMCA is used in this way; for example, although three classes may be independently modelled, a sample is assigned to the class whose distance is closest to the unknown, although the originators did not intend it to be employed in this manner.…”

“…Further details have been described elsewhere [16][17][18][19][20][21]. The dataset consists of the thermal profiles of 293 samples, involving monitoring the change in physical properties as they are heated.…”

“…One has a high D but low Q as it still falls on the line but is far from the centre and another has a low D but high Q as it does not fall onto the line but its projection would place it in the centre of the class. It is possible to combine the Q and D statistics numerically [21]. One approach is to determine the fit of the ith sample to the model for the Ath class.…”

One class classifiers for process monitoring illustrated by the application to online HPLC of a continuous process

“…Support vector domain description (SVDD) [11,[15][16][17][39][40][41]] is a modified version of support vector machines (SVMs) [42][43][44][45][46][47]. The difference between SVDD and SVMs is that the latter method computes a boundary between two or more classes, whereas, for SVDD, a boundary is drawn around each individual class separately resulting in a separate (and different) model for each class; therefore, it is sometimes called a 'one class classifier' .…”

“…The mathematical basis of SVDD is well described elsewhere [11,[15][16][17][39][40][41]. In this paper, a radial basis function (RBF) was used for the kernel since it requires only one parameter to define it [40][41].…”