The overfitted brain: Dreams evolved to assist generalization

Abstract

Understanding of the evolved biological function of sleep has advanced considerably in the past decade. However, no equivalent understanding of dreams has emerged. Contemporary neuroscientific theories often view dreams as epiphenomena, and many of the proposals for their biological function are contradicted by the phenomenology of dreams themselves. Now, the recent advent of deep neural networks (DNNs) has finally provided the novel conceptual framework within which to understand the evolved function of dreams. Notably, all DNNs face the issue of overfitting as they learn, which is when performance on one dataset increases but the network's performance fails to generalize (often measured by the divergence of performance on training versus testing datasets). This ubiquitous problem in DNNs is often solved by modelers via ''noise injections'' in the form of noisy or corrupted inputs. The goal of this paper is to argue that the brain faces a similar challenge of overfitting and that nightly dreams evolved to combat the brain's overfitting during its daily learning. That is, dreams are a biological mechanism for increasing generalizability via the creation of corrupted sensory inputs from stochastic activity across the hierarchy of neural structures. Sleep loss, specifically dream loss, leads to an overfitted brain that can still memorize and learn but fails to generalize appropriately. Herein this ''overfitted brain hypothesis'' is explicitly developed and then compared and contrasted with existing contemporary neuroscientific theories of dreams. Existing evidence for the hypothesis is surveyed within both neuroscience and deep learning, and a set of testable predictions is put forward that can be pursued both in vivo and in silico. ll

Grounding deep neural network predictions of human categorization behavior in understandable functional features: The case of face identity

“…Their discriminative goals can be solved with shortcuts 16 relying on a few isolated features, which are not sufficient to generalize as humans do. 76 This aligns with previous findings that multi-task learning 77 , 78 , 79 and generative models 80 enhance robustness against adversarial attacks and best predict behavior under severe testing. 17 In relation to faces as a broad category, future research could systematically study the number and combinatorics of categorizations (e.g., identity, sex, age, ethnicity, facial expressions) and rendering factors (e.g., illumination, pose, occlusions) that would be required to enhance the latent spaces to match (or surpass) the predictiveness of behavior of the latent space of the viAE, also across varying levels of familiarity.…”

Balancing Prediction and Surprise: A Role for Active Sleep at the Dawn of Consciousness?

“…It is possible that active sleep -and the rehearsal of narratives and contingencies accumulated during the dayis similarly in the service of removing redundant connections and thereby minimizing complexity. Cognitive flexibility could thus be seen as emanating from processes that preclude overfitting overly parametrized internal models (with redundant and exuberant synaptic connections) (Hoel, 2021). This view would tie neatly with the synaptic homeostasis hypothesis that has been attributed to deep sleep in higher animals (Huber et al, 2004;Tononi and Cirelli, 2006).…”

Relationship Between Epilepsy and Dreaming: Current Knowledge, Hypotheses, and Perspectives

“…Although dream function remains a matter of debate, the role of dreams in the regulation of emotions and in the consolidation of memory has been widely emphasized ( Malinowski and Horton, 2015 ; Vallat et al, 2017a ; Plailly et al, 2019 ; Scarpelli et al, 2019 ). Other theories propose that dreams may act as virtual reality training (to notably improve social and defensive skills) to prepare for waking life challenges ( Valli et al, 2005 ; Hobson, 2009 ; Hobson et al, 2014 ; Tuominen et al, 2019 ) or that they may prevent the brain from overfitting waking experiences by allowing generalization of learned neural representations aiming to ensure a better adaptation to real-world life ( Wagner et al, 2004 ; Hoel, 2021 ). Altered dreaming processes may thus have consequences on epilepsy patients’ ability to adapt to waking life demands.…”