Toward Human-Scale Brain Computing Using 3D Wafer Scale Integration

Abstract

The Von Neumann architecture, defined by strict and hierarchical separation of memory and processor, has been a hallmark of conventional computer design since the 1940s. It is becoming increasingly unsuitable for cognitive applications, which require massive parallel processing of highly interdependent data. Inspired by the brain, we propose a significantly different architecture characterized by a large number of highly interconnected simple processors intertwined with very large amounts of low-latency memory. We contend that this memory-centric architecture can be realized using 3D wafer scale integration for which the technology is nearing readiness, combined with current CMOS device technologies. The natural fault tolerance and lower power requirements of neuromorphic processing make 3D wafer stacking particularly attractive. In order to assess the performance of this architecture, we propose a specific embodiment of a neuronal system using 3D wafer scale integration; formulate a simple model of brain connectivity including short-and long-range connections; and estimate the memory, bandwidth, latency, and power requirements of the system using the connectivity model. We find that 3D wafer scale integration, combined with technologies nearing readiness, offers the potential for scaleup to a primate-scale brain, while further scaleup to a human-scale brain would require significant additional innovations.

Magnetization reversal in bent nanofibers of different cross sections

“…The corresponding snapshots of the magnetization reversal processes are depicted in Figs. [12][13][14]. Depending on the deflection, more or less orientation into the y direction (i.e.…”

“…One possibility to create such a new "cognitive" computer hardware is based on a large number of highly interconnected simple processors, connected with large amounts of lowlatency memory, giving thus up the classical von Neumann architecture with the strict separation of memory and processor. 13 For example, Grollier et al propose networks of connected memristors to realize bioinspired or "neuromorphic" computing. 14 Another possibility is based on creating bio-inspired nanofiber networks of fibers prepared in a form of mats, partly mimicking the brain's geometry and its data storage, processing and conducting properties.…”

A Case for 3D Integrated System Design for Neuromorphic Computing and AI Applications

“…Kursun (2010), micro-channels and other cooling solutions may be challenging to implement in advanced 3D stacks due to the layer thicknesses A. Kumar (2017). In such many-layer 3D stacks, bonding materials and BEOL layers (with different thermal characteristics than silicon layers) E. Kursun (2012), macro placement and activity management require special design considerations.…”

Power Delivery for Silicon Interconnect Fabric

“…These fine-feature technologies support integration of multiple ICs in high proximity in the horizontal, vertical, or both dimensions, leading to a significant reduction in the latency and energy of inter-die communication. Moreover, heterogeneous integration enables novel applications such as a brain-scale neural network (10 11 neurons and 10 14 synapses) on a 3-D wafer-scale platform [6], implantable medical devices on a bio-compatible flexible platform [7], and low-energy small form factor Internet-of-Things (IoT) systems on 3-D ICs [8].…”