Learning in Ultrametric Committee Machines

Abstract

Learning in ultrametric committee machines the date of receipt and acceptance should be inserted later Abstract The problem of learning by examples in ultrametric committee machines (UCMs) is studied within the framework of statistical mechanics. Using the replica formalism we calculate the average generalization error in UCMs with L hidden layers and for a large enough number of units. In most of the regimes studied we find that the generalization error, as a function of the number of examples presented, develops a discontinuous drop at a critical value of the load parameter. We also find that when L > 1 a number of teacher networks with the same number of hidden layers and different overlaps induce learning processes with the same critical points.

Storage capacity of ultrametric committee machines

“…A divergence in Σ ℓ occurs if the overlap ζ ℓ is too large (particularly if the scaling law given by (5) is not respected). As it was observed in [17,18], too large an overlap effectively reduces the complexity of the machine, i.e. the UCM can be effectively replaced by another UCM with less hidden layers, rendering the subsequent calculations meaningless.…”

“…Recently, more complex architectures have drawn attention, due to the possibility to obtain analytically some computational properties [17,18]. In this article we are interested in computing the storage capacity of such architectures, known as ultrametric committee machines (UCMs).…”