Information Fragmentation, Encryption and Information Flow in Complex Biological Networks

Abstract

Assessing where and how information is stored in biological networks (such as neuronal and genetic networks) is a central task both in neuroscience and in molecular genetics, but most available tools focus on the network’s structure as opposed to its function. Here, we introduce a new information-theoretic tool—information fragmentation analysis—that, given full phenotypic data, allows us to localize information in complex networks, determine how fragmented (across multiple nodes of the network) the information is, and assess the level of encryption of that information. Using information fragmentation matrices we can also create information flow graphs that illustrate how information propagates through these networks. We illustrate the use of this tool by analyzing how artificial brains that evolved in silico solve particular tasks, and show how information fragmentation analysis provides deeper insights into how these brains process information and “think”. The measures of information fragmentation and encryption that result from our methods also quantify complexity of information processing in these networks and how this processing complexity differs between primary exposure to sensory data (early in the lifetime) and later routine processing.

Detecting Information Relays in Deep Neural Networks

“…Interestingly, when this process arrives at a set of nodes that taken together is essential in relaying the information, it can happen that the removal of any of the nodes of the set causes the remaining neurons to have . Information in such an essential set can be seen to be encrypted , to the point where no node can be removed without losing all of the information [ 38 ]. However, this creates a situation in which the last nodes, when removed, appear to not contribute any information, even though they are essential.…”

“…If catastrophic forgetting is due to a lack of modularization of information, it becomes crucial to accurately measure this modularization to identify learning schemes that promote modules. The problem of identifying modules responsible for different functions is further aggravated when information theory and perturbation analysis (via node knockout) disagree [ 38 , 39 ].…”

“…However, if information is stored redundantly , perturbing only part of the redundant nodes will not necessarily disrupt function, even though they carry information. At the same time, nodes without function or information can still accidentally perturb outputs when experiencing noise [ 38 , 39 ].…”

Using artificial intelligence to fit, compare, evaluate, and discover computational models of decision behavior

“…Information theoretic approaches complement these correlational analysis, and have been used to characterize cognitive and computational systems alike (Tononi, 2004;Marstaller et al, 2013;Tehrani-Saleh & Adami, 2020;Hintze & Adami, 2023), and can also be combined with perturbation analysis (Bohm et al, 2022;Hintze & Adami, 2022).…”

Neuroevolution gives rise to more focused information transfer compared to backpropagation in recurrent neural networks

“…If the energy landscape of the solution space is rugged, the different processes could give rise to different endpoints (O) models of the environment-are stored in evolved computational systems [12,13]. In humans [14][15][16] as well as in computational models such as Hierarchical Temporal Memory [17] and Markov Brains [12], these memories are sparsely distributed, meaning that small groups of neurons carry the information associated with complex concepts [18]. Preliminary evidence suggests that memories in RNNs appear to be much more distributed over nearly all the nodes of the network [13].…”