Neural Modularity Helps Organisms Evolve to Learn New Skills without Forgetting Old Skills

Abstract

A long-standing goal in artificial intelligence is creating agents that can learn a variety of different skills for different problems. In the artificial intelligence subfield of neural networks, a barrier to that goal is that when agents learn a new skill they typically do so by losing previously acquired skills, a problem called catastrophic forgetting. That occurs because, to learn the new task, neural learning algorithms change connections that encode previously acquired skills. How networks are organized critically affects their learning dynamics. In this paper, we test whether catastrophic forgetting can be reduced by evolving modular neural networks. Modularity intuitively should reduce learning interference between tasks by separating functionality into physically distinct modules in which learning can be selectively turned on or off. Modularity can further improve learning by having a reinforcement learning module separate from sensory processing modules, allowing learning to happen only in response to a positive or negative reward. In this paper, learning takes place via neuromodulation, which allows agents to selectively change the rate of learning for each neural connection based on environmental stimuli (e.g. to alter learning in specific locations based on the task at hand). To produce modularity, we evolve neural networks with a cost for neural connections. We show that this connection cost technique causes modularity, confirming a previous result, and that such sparsely connected, modular networks have higher overall performance because they learn new skills faster while retaining old skills more and because they have a separate reinforcement learning module. Our results suggest (1) that encouraging modularity in neural networks may help us overcome the long-standing barrier of networks that cannot learn new skills without forgetting old ones, and (2) that one benefit of the modularity ubiquitous in the brains of natural animals might be to alleviate the problem of catastrophic forgetting.

Modular Brain Network Organization Predicts Response to Cognitive Training in Older Adults

“…While previous studies have shown that neural factors (e.g., frontal alpha power and striatal volume) are related to skill learning [48–51], the aspects of brain structure and function that predicted learning were variable across studies. Computational models examining the modularity of neural networks have demonstrated that more modular networks enable organisms to learn new skills without forgetting old ones [52]. Further, greater segregation of visual and motor sub-networks (i.e., more modular sub-networks) is predictive of motor learning [53] and the segregation of these sub-networks increases over the course of learning [54].…”

Modular Brain Networks

“…In the face of unpredictable endogenous or exogenous changes, swapping or rearranging maladaptive modules is less costly than redesigning the entire system. In a similar vein, Ellefsen et al (2015) demonstrated that modular brain networks can help prevent catastrophic forgetting, that is, the loss of a previously learned skill upon learning a new one.…”

Morphological Modularity Can Enable the Evolution of Robot Behavior to Scale Linearly with the Number of Environmental Features

“…First, modular systems possess a form of robustness: modular systems can more rapidly adapt to certain kinds of changes in their environments, compared to non-modular systems. Second, modular neural networks are better able to avoid catastrophic forgetting than non-modular networks (Ellefsen et al, 2015). Catastrophic forgetting (French, 1999) is a common problem in machine learning, whereby a learner must forget something in order to learn something new.…”