The unreasonable effectiveness of deep learning in artificial intelligence

Abstract

Deep learning networks have been trained to recognize speech, caption photographs and translate text between languages at high levels of performance. Although applications of deep learning networks to real world problems have become ubiquitous, our understanding of why they are so effective is lacking. These empirical results should not be possible according to sample complexity in statistics and non-convex optimization theory. However, paradoxes in the training and effectiveness of deep learning networks are being investigated and insights are being found in the geometry of high-dimensional spaces. A mathematical theory of deep learning would illuminate how they function, allow us to assess the strengths and weaknesses of different network architectures and lead to major improvements. Deep learning has provided natural ways for humans to communicate with digital devices and is foundational for building artificial general intelligence. Deep learning was inspired by the architecture of the cerebral cortex and insights into autonomy and general intelligence may be found in other brain regions that are essential for planning and survival, but major breakthroughs will be needed to achieve these goals.

Artificial intelligence for clinical decision support in neurology

“…However, arguably the biggest advances in AI to date have come in the last decade, as massive scale data and hardware suitable to process these data have become available, and sophisticated deep-learning methods— that aim to imitate the working of the human brain in processing data —became computationally feasible ( Ngiam et al , 2011 ; LeCun et al , 2015 ; Schmidhuber, 2015 ; Goodfellow et al , 2016 ) . Deep learning is now widely regarded as the foundation of contemporary AI ( Sejnowski, 2020 ) ( Fig. 1 and Box 1 ).…”

A spiking neural program for sensory-motor control during foraging in flying insects

“…Deep artificial neural networks (62) have demonstrated great success over the recent years. Particularly, in the domains of image recognition, natural language processing and deep reinforcement learning (63). Despite their success, when applied to agent-based systems, their major drawback becomes evident.…”

The best laid plans: Computational principles of ACC

“…Meanwhile, influential theories of human information processing, based loosely on biological neural networks, precipitated the development in AI of artificial neural networks [91], which ultimately paved the way for advances in machine learning that underpin a range of modern-day technologies including object recognition algorithms [92]. In turn, the new algorithms provided insight into the neural mechanisms of visual perception [93,94] and other cognitive functions.…”