Collective intelligence for deep learning: A survey of recent developments

Abstract

In the past decade, we have witnessed the rise of deep learning to dominate the field of artificial intelligence. Advances in artificial neural networks alongside corresponding advances in hardware accelerators with large memory capacity, together with the availability of large datasets enabled practitioners to train and deploy sophisticated neural network models that achieve state-of-the-art performance on tasks across several fields spanning computer vision, natural language processing, and reinforcement learning. However, as these neural networks become bigger, more complex, and more widely used, fundamental problems with current deep learning models become more apparent. State-of-the-art deep learning models are known to suffer from issues that range from poor robustness, inability to adapt to novel task settings, to requiring rigid and inflexible configuration assumptions. Collective behavior, commonly observed in nature, tends to produce systems that are robust, adaptable, and have less rigid assumptions about the environment configuration. Collective intelligence, as a field, studies the group intelligence that emerges from the interactions of many individuals. Within this field, ideas such as self-organization, emergent behavior, swarm optimization, and cellular automata were developed to model and explain complex systems. It is therefore natural to see these ideas incorporated into newer deep learning methods. In this review, we will provide a historical context of neural network research’s involvement with complex systems, and highlight several active areas in modern deep learning research that incorporate the principles of collective intelligence to advance its capabilities. We hope this review can serve as a bridge between the complex systems and deep learning communities.

The scaling of goals from cellular to anatomical homeostasis: an evolutionary simulation, experiment and analysis

“…One purpose of these kinds of simulations is to develop protocols that could enable robotics and artificial intelligence approaches to benefit from the kind of robustness and problem-solving ability observed in biology. In artificial intelligence, the fact that all intelligences are collective intelligences has not been emphasized [7], but there are recent attempts for swarm robotics and deep learning to develop new methods based on collective intelligence [48,103]. Deep learning has started to integrate the approach with adversarial networks with two networks working together [104].…”

Humans, robots and artificial intelligences reconfiguring urban life in a crisis

“…Current debates about the impact of AI and their relationship with humans have generally coalesced around three broad perspectives (Peeters et al, 2021)-a technology-centric perspective claiming that AI will outperform and potentially displace humans, a human-centric perspective claiming that humans will always remain superior to AI and lastly a perspective centered on collective intelligence-ultimately, the case of MK suggests that assemblages of human and non-human intelligences have potential to lead to the emergence of cognitive capabilities greater than the sum of its parts (Ha and Tang, 2022). Crucially, the resulting forms of collective intelligence cannot be reduced to some combination of bits in a computer and synapses in human brains.…”

General intelligence requires rethinking exploration

“…There remain many open questions on which aspects of intelligence cannot be adequately captured by existing approaches, e.g. robust symbolic reasoning [47], causal understanding [48,49], hierarchical planning in a world model [50] and large-scale collective behaviours like cultural learning [51][52][53]. We argue that, even if provided a joint model and optimization scheme that captures these missing aspects of intelligence, the training data itself remains a fundamental limitation, and addressing it requires royalsocietypublishing.org/journal/rsos R. Soc.…”