An activation reinforcement based classifier system for balancing generalisation and specialisation (ARCS)

Abstract

Learning Classifier Systems are reinforcement-based learning systems that allow the development of generalised rule sets. They allow a balance between higher level generalisable learning and reinforcement, and have been used in a number of systems to introduce principles from psychology to guide methods of learning. A classifier system based on Activation Reinforcement (ARCS) is described, based on accessibility of traces in semantic memory, that provides a strength related learning technique allowing balance of generalisation and specialisation of rules. The system is based on a minimal number of design principles, is arguably simpler in design than existing classifier systems, and has clearer connections with human cognitive models. Performance on the standard Woods environments shows fast, stable learning allowing near-optimal behaviour. The methods used by ARCS have a number of differences with existing ZCS and XCS approaches, that have potential advantages regarding generalisation techniques, and for addressing sparsely rewarded domains.

Coarse and fine learning in deep networks

“…Learning Classifier Systems use a form of reinforcement-based learning, both in terms of Q-learning style Reinforcement Learning to update predictions on multistep problems, and Psychology related reinforcement to influence maintenance of the population 2 , and as such it is possible to view these algorithms in the context of cognitive processes. Connections between Reinforcement Learning methods and cognitive processes are well established [15], and there are further similarities between the population reinforcement methods used and models of abstract cognitive processes, such as reinforcement of memory traces [16], [10]. The term 'Evolutionary Computing' is commonly used to describe this family of techniques, and the terminology will be maintained to refer to LCS and related systems, however the evolutionary paradigm is not significant for this study.…”

“…Common features are captured in building blocks, and are preserved through crossover in the genetic algorithm. Classifiers can also be defined using a hierarchical structure, employing a population of reused features, where features are recorded discretely instead of redundantly, and recombined in various combinations to produce classification rules [16]. The hierarchical approach maintains features according to reinforcement, and creates new features based on combinations of existing ones, without the use of a genetic algorithm.…”

Formation and activation of feature hierarchies under reinforcement

“…The system used in this study for examining hierarchical representations is based on the Activation Reinforcement Classifier System [33]. The general design of the system is shown in Figure 1.…”

Learning feature hierarchies under reinforcement

“…A form of learning classifier system, the Activation-Reinforcement Classifier System [11] uses a design based on models of reinforcement of memory traces [12], [13], with the intention of building links with abstract cognitive processes. The system separates scalar reinforcement of rules from expected reward, which should provide advantages of allowing use of re-used elements which participate in multiple rules with different outcomes.…”