Immune networks: multitasking capabilities near saturation

Abstract

Abstract. Pattern-diluted associative networks were introduced recently as models for the immune system, with nodes representing T-lymphocytes and stored patterns representing signalling protocols between T-and B-lymphocytes. It was shown earlier that in the regime of extreme pattern dilution, a system with N T Tlymphocytes can manage a number N B = O(N δ T ) of B-lymphocytes simultaneously, with δ < 1. Here we study this model in the extensive load regime N B = αN T , with also a high degree of pattern dilution, in agreement with immunological findings. We use graph theory and statistical mechanical analysis based on replica methods to show that in the finite-connectivity regime, where each T-lymphocyte interacts with a finite number of B-lymphocytes as N T → ∞, the T-lymphocytes can coordinate effective immune responses to an extensive number of distinct antigen invasions in parallel. As α increases, the system eventually undergoes a second order transition to a phase with clonal cross-talk interference, where the system's performance degrades gracefully. Mathematically, the model is equivalent to a spin system on a finitely connected graph with many short loops, so one would expect the available analytical methods, which all assume locally tree-like graphs, to fail. Yet it turns out to be solvable. Our results are supported by numerical simulations.

Emergence of Compositional Representations in Restricted Boltzmann Machines

“…2) provided that nonlinear Φ (ReLU) and appropriate threshold values θ are considered. The presence of the fields g i acting on the visible units (absent in the v i = ±1 model of [17][18][19]), is also crucial for the existence of our compositional phase as explained above.It would be interesting to extend our work to more than one layers of hidden units, or to other types of nonlinear Φ. While numerical studies of RBMs with Bernoulli hidden units show no qualitative change compared to ReLU, choosing Φ(h) growing asymptotically faster than h could affect the nature of the extracted features [23].…”

“…In this framework magnetized hidden units identify retrieved patterns, and α corresponds to the capacity of the autoassociative memory. Tsodyks and Feigel'man showed how the critical capacity (for single pattern retrieval) could be dramatically increased with sparse weights (p 1) and appropriate tuning of the fields g i [21]; however this effect could be achieved only with vanishingly low activities q. Agliari and collaborators showed in a series of papers [17,18] that multiple sparse patterns could be simultaneously retrieved in the case of linear Φ and vanishing capacity α = 0 (finite M ). Finite capacity α ∼ c −2 could be achieved at zero 5 temperature in the limit of extreme sparsity, p = c/N , only [19]; for MNIST p 0.1 and N = 784 would give α ∼ 2.10 −4 .…”

“…2(d)) measures the fraction of nonzero weights, p = i p i /N . This distribution was previously introduced to study parallel storage of multiple sparse items in the Hopfield model [17,18]. For simplicity the fields on visible units and the potentials acting on hidden units are chosen to be uniform, g i = g and U µ = U ReLU ( Fig.…”

Cancer-driven dynamics of immune cells in a microfluidic environment

“…From the theoretical counterpart, models coming from mathematics or theoretical physics, such as maximum entropy principle4, disordered statistical mechanics56, complex optimization7, graph theory8, stochastic processes9 and dynamical systems10 are being adapted to biological systems, allowing a more complete picture of cellular behavior from advanced imaging studies. In parallel, the development of methods aimed at considering the system as a whole, hence with all its constituents mutually interacting, have become a major topic, involved as a necessary step beyond reductionism limitations6111213.…”

“…In parallel, the development of methods aimed at considering the system as a whole, hence with all its constituents mutually interacting, have become a major topic, involved as a necessary step beyond reductionism limitations6111213. For example, cancer progression involves multiple events and is the result of the interactions with cells of the immune system within the tumor micro-environment1415161718192021.…”