Paradoxical replay can protect contextual task representations from destructive interference when experience is unbalanced

Abstract

Experience replay is a powerful mechanism to learn efficiently from limited experience. Despite several decades of compelling experimental results, the factors that determine which experiences are selected for replay remain unclear. A particular challenge for current theories is that on tasks that feature unbalanced experience, rats paradoxically replay the less-experienced trajectory. To understand why, we simulated a feedforward neural network with two regimes: rich learning (structured representations tailored to task demands) and lazy learning (unstructured, task-agnostic representations). Rich, but not lazy, representations degraded following unbalanced experience, an effect that could be reversed with paradoxical replay. To test if this computational principle can account for the experimental data, we examined the relationship between paradoxical replay and learned task representations in the rat hippocampus. Strikingly, we found a strong association between the richness of learned task representations and the paradoxicality of replay. Taken together, these results suggest that paradoxical replay specifically serves to protect rich representations from the destructive effects of unbalanced experience, and more generally demonstrate a novel interaction between the nature of task representations and the function of replay in artificial and biological systems.