Neural network constructive algorithms: Trading generalization for learning efficiency?

Śmieja, F. J.

doi:10.1007/bf01189880

Cited by 56 publications

(12 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Different constructive learning algorithms allow trading off certain performance measures (e.g., learning time) for others (e.g., network size and generalization accuracy) [47].…”

Section: • Estimation Of Expected Case Complexity Of the Learningmentioning

confidence: 99%

“…These algorithms differ in terms of their choices regarding: restrictions on input representation (e.g., binary or bipolar valued inputs); when to add a neuron; where to add a neuron; connectivity of the added neuron; weight initialization for the added neuron; how to train the added neuron (or a subnetwork affected by the addition); and so on. They can be shown to converge to networks which yield zero classification errors on any noncontradictory training set involving two output classes (see [47] for a unifying framework that explains the convergence properties of these constructive algorithms). A geometrical analysis of the decision boundaries of some of these algorithms is presented in [7].…”

Section: ) Constructive Learning Using Iterative Weight Updatementioning

confidence: 99%

“…The geometric approach to constructive learning can be applied successfully in solving small to medium scale problems. However, the global search strategy employed by these algorithms can pose a limitation when learning from very large training sets [47]. Further, the reliance on a suitably chosen distance metric (in the case of GAL and DistAl) makes it imperative for the user to try out a variety of distance metrics for each learning problem.…”

Section: ) Constructive Learning Using Iterative Weight Updatementioning

confidence: 99%

See 2 more Smart Citations

Constructive neural-network learning algorithms for pattern classification

Parekh

Yang

Honavar

2000

IEEE Trans. Neural Netw.

209

108

View full text Add to dashboard Cite

Abstract-Constructive learning algorithms offer an attractive approach for the incremental construction of near-minimal neural-network architectures for pattern classification. They help overcome the need for ad hoc and often inappropriate choices of network topology in algorithms that search for suitable weights in a priori fixed network architectures. Several such algorithms are proposed in the literature and shown to converge to zero classification errors (under certain assumptions) on tasks that involve learning a binary to binary mapping (i.e., classification problems involving binary-valued input attributes and two output categories). We present two constructive learning algorithms MPyramid-real and MTiling-real that extend the pyramid and tiling algorithms, respectively, for learning real to M-ary mappings (i.e., classification problems involving real-valued input attributes and multiple output classes). We prove the convergence of these algorithms and empirically demonstrate their applicability to practical pattern classification problems. Additionally, we show how the incorporation of a local pruning step can eliminate several redundant neurons from MTiling-real networks.

show abstract

“…Different constructive learning algorithms allow trading off certain performance measures (e.g., learning time) for others (e.g., network size and generalization accuracy) [47].…”

Section: • Estimation Of Expected Case Complexity Of the Learningmentioning

confidence: 99%

Section: ) Constructive Learning Using Iterative Weight Updatementioning

confidence: 99%

Section: ) Constructive Learning Using Iterative Weight Updatementioning

confidence: 99%

See 1 more Smart Citation

Constructive neural-network learning algorithms for pattern classification

Parekh

Yang

Honavar

2000

IEEE Trans. Neural Netw.

209

108

View full text Add to dashboard Cite

show abstract

“…Valle (2005) presents various approaches to built smart adaptive devices. There still some topics as hardware friendly learning algorithms, as pertubation learning (Jabri and Flower, 1991), constructive learning (Smieja, 1993), cascade error projection learning (Duong, 1995;Duong and Stubberud, 1995), and local learning (Chen et al, 2000). Some HNN based on Multilayer Perceptron (MLP) (D'Acierno, 2000;Kumar et al, 1994), radial basis function (Fakhraie et al, 1994;Verleysen et al, 1994;Yang and Paindavoine, 2005), and Neocognition (Patnaik and Rao, 2000) and neurocomputers (Glesner and Poechmueller, 1994;Strey and Avellana, 2000).…”

Section: Introductionmentioning

confidence: 99%

Adaptive Boolean Logic Using Ferroelectrics Capacitors as Basic Units of Artificial Neurons

Silva¹,

Leite²,

Guerreiro³

et al. 2011

Ferroelectrics - Applications

View full text Add to dashboard Cite

“…-Trade-offs among performance measures. Different constructive learning methods allow tradeoffs in certain performance measures, such as tradeoffs between learning time and accuracy [122]. -Incorporation of prior knowledge.…”

Section: Cooperative Approach In Evolutionary Computationmentioning

confidence: 99%

Scaling up data mining algorithms: review and taxonomy

García-Pedrajas

Haro-García

2012

Prog Artif Intell

View full text Add to dashboard Cite

The overwhelming amount of data that are now available in any field of research poses new problems for data mining and knowledge discovery methods. Due to this huge amount of data, most of the current data mining algorithms are inapplicable to many real-world problems. Data mining algorithms become ineffective when the problem size becomes very large. In many cases, the demands of the algorithm in terms of the running time are very large, and mining methods cannot be applied when the problem grows. This aspect is closely related to the time complexity of the method. A second problem is linked with performance; although the method might be applicable, the size of the search space prevents an efficient execution, and the resulting solutions are unsatisfactory. Two approaches have been used to deal with this problem: scaling up data mining algorithms and data reduction. However, because data reduction is a data mining task itself, this technique also suffers from scalability problems. Thus, for many problems, especially when dealing with very large datasets, the only way to deal with the aforementioned problems is to scale up the data mining algorithm. Many efforts have been made to obtain methods that can be used to scale up existing data mining algorithms. In this paper, we review the methods that have been used to address the problem of scalability. We focus on general ideas, rather than specific implementations, that can be used to provide a general view of the current approaches for scaling up data mining methods. A taxonomy of the algorithms is proposed, and many examples of different tasks are presented. Among the different techniques used for data mining, we will pay special attention to evolutionary methods, because these methods have been used very successfully in many data mining tasks.

show abstract

Neural network constructive algorithms: Trading generalization for learning efficiency?

Cited by 56 publications

References 13 publications

Constructive neural-network learning algorithms for pattern classification

Constructive neural-network learning algorithms for pattern classification

Adaptive Boolean Logic Using Ferroelectrics Capacitors as Basic Units of Artificial Neurons

Scaling up data mining algorithms: review and taxonomy

Contact Info

Product

Resources

About