Comparing different neural network architectures for classifying handwritten digits

Multiplier-free feedforward networks

“…The numeral database 6 used for these simulations was collected at Bell Labs [3]. It consists of 1200 isolated handwritten samples of numerals 0..9.…”

“…This database was then carved into two subsets of 600 samples each -the first five samples of each numeral from every writer were used for training and the rest for testing. Guyon et al [3], and Druker and Le Cun [2], both have reported a generalization performance of 97% with a 256:20:10 network trained using conventional BP, and a 256:40:10 network trained using 'double backpropagation', respectively. For the simulations presented here, the 256 element matrices were transformed into 32 element vectors by summing the rows and columns.…”

“…For the simulations presented here, the 256 element matrices were transformed into 32 element vectors by summing the rows and columns. Although this 8-fold reduction in input dimension did cause a 4% reduction in generalization performance (compared with [2], [3]), it made the running of many more simulations possible due to the reduced memory and CPU requirements. Each element of these 32-element vectors was then transformed as…”

“…The first set has both discrete and continuous inputs, may have some irrelevant inputs, may have much noise in the inputs, may have a high degree of correlation between inputs, and has a single binary output [7], [9]. The second set is a representative of the image processing applications, has a high degree of information redundancy, has discrete inputs, and many binary outputs [2], [3].…”

“…Two data sets were used for these experiments. Boththe 'onset of diabetes prediction' data set and the handwritten numeral recognition database -are publicly available and have been used in the past by many for the purpose of benchmarking [2], [3], [7], [9]. The first set has both discrete and continuous inputs, may have some irrelevant inputs, may have much noise in the inputs, may have a high degree of correlation between inputs, and has a single binary output [7], [9].…”