Symbolic and neural learning algorithms: An experimental comparison

Abstract

Abstract. Despite the fact that many symbolic and neural network (connectionist) learning algorithms address the same problem of learning from classified examples, very little is known regarding their comparative strengths and weaknesses. Experiments comparing the ID3 symbolic learning algorithm with the perceptron and backpropagation neural learning algorithms have been performed using five large, real-world data sets, Overall, backpropagation performs slightly better than the other two algorithms in terms of classification accuracy on new examples, but takes much longer to train. Experimental results suggest that backpropagation can work significantly better on data sets containing numerical data. Also analyzed empirically are the effects of (1) the amount of training data, (2) imperfect training examples, and (3) the encoding of the desired outputs. Backpropagation occasionally outperforms the other two systems when given relatively small amounts of training data. It is slightly more accurate than ID3 when examples are noisy or incompletely specified. Finally, backpropagation more effectively utilizes a "distributed" output encoding.

A weighted nearest neighbor algorithm for learning with symbolic features

“…For comparison, we give results from the NETtalk program, which used the back propagation algorithm. Shavlik et al (1989) replicated Sejnowski and Rosenberg's methodology as ,part of their work, and although their results differ from Sejnowski and Rosenberg's (not surprisingly, since back propagation networks require much tuning), they make for easier comparison with ours. This property follows from the fact that the original Sejnowski and Rosenberg study used a distributed output encoding; that is, their system produced a 26-bit sequence (rather than one bit for each of the 115 phoneme/stress combinations).…”

“…These domains have received considerable attention from connectionist researchers who employed the back propagation learning algorithm (Sejnowski & Rosenberg, 1986;Qian & Sejnowski, 1988;Towell et al, 1990). In addition, the word pronunciation problem has been the subject of a number of comparisons using other machine learning algorithms (Stanfill & Waltz, 1986;Shavlik et al, 1989;Dietterich et al, 1990). All of these domains represent problems of considerable practical importance, and all have symbolic feature values, which makes them difficult for conventional nearest neighbor algorithms.…”

“…Training is normally performed by repeatedly presenting the network with instances from a training set, and allowing it gradually to converge on the best set of weights for the task, using (for example) the back propagation algorithm. Weiss and Kapouleas (1989), Mooney et al (1989), and Shavlik et al (1989) report that back propagation's training time is many orders of magnitude greater than training time for algorithms such as ID3, frequently by factors of 100 or more. In addition, neural net algorithms have a number of parameters (e.g., the "momentum" parameter) that need to be tweaked by the programmer, and may require much additional time.…”

“…We chose for our comparisons three domains that have received considerable attention from the machine learning research community: the word-pronunciation task (Sejnowski & Rosenberg, 1986;Shavlik et al, 1989), the prediction of protein secondary structure (Qian & Sejnowski, 1988;Holley & Karplus, 1989), and the prediction of DNA promoter sequences . Each domain has only symbolic-valued features; thus, our MVDM is applicable whereas standard Euclidean distance is not.…”

Encouraging experimental results on learning CNF

“…The same version was used in the experiments reported by Shavlik et al (1991). It learns a single tree for classifying examples into rnultiple categories and uses the normal information-gain splitting criterion.…”