Choice effect of linear separability testing methods on constructive neural network algorithms: An empirical study

A convergent algorithm for a cascade network of multiplexed dual outputdiscrete perceptrons for linearly nonseparable classification

“…On the other hand, the PA [3], which uses the PLR, can find weights providing the minimum output error for linearly nonseparable problems. In the literature, a comparison of different linear separability testing methods on constructive neural networks was studied [18]. The results of the experiments done on 9 different data sets and by using 6 different methods are summarized in Table 2.…”

“…Comparing the sizes of the constructed networks in Table 3, it can be concluded that the recursive deterministic perceptron (RDP) network [18], which applies a specific construction scheme, produces the worst results for the considered data sets in terms of the network sizes if the PLR is used at each step of the construction. As explained in [18], the lack of a procedure for determining a proper stopping time for the PLR due to the linearly nonseparable nature of the problem may be the reason. However, to the knowledge of the authors of this paper, there is no theoretical result in the literature to allow comparison of the performance of PLR versus the other methods used at each step of constructive networks for obtaining a suboptimal linear separation.…”

“…As listed in Table 3, a set of different methods are investigated for linearly nonseparable learning problems and their performances are compared in a constructive neural network algorithm [18]. As summarized in Table 2, although there are no stopping criteria established, the PLR method is found to be among the fast methods in [18]. Table 3.…”

“…Results obtained with the 6 models, using different methods for testing linear separability, and 9 data sets in terms of the constructed network size ( m represents the average over the data sets, and max is the maximum constructed network size) [18]. Comparing the sizes of the constructed networks in Table 3, it can be concluded that the recursive deterministic perceptron (RDP) network [18], which applies a specific construction scheme, produces the worst results for the considered data sets in terms of the network sizes if the PLR is used at each step of the construction. As explained in [18], the lack of a procedure for determining a proper stopping time for the PLR due to the linearly nonseparable nature of the problem may be the reason.…”

“…For instance, instead of the PA, there is the thermal perceptron learning algorithm [20], which is a modified PLR with an annealing factor as exploited in other constructive algorithms [21]. As listed in Table 3, a set of different methods are investigated for linearly nonseparable learning problems and their performances are compared in a constructive neural network algorithm [18]. As summarized in Table 2, although there are no stopping criteria established, the PLR method is found to be among the fast methods in [18].…”