SYNAPSE-1: a high-speed general purpose parallel neurocomputer system

Ramacher, Ulrich; Raab, W.; Hachmann, J.A.U.; Beichter, J.; Brüls, N.; Wesseling, M.; Sicheneder, E.; Gläß, J.; Wurz, A.; Männer, Reinhard

doi:10.1109/ipps.1995.395862

Cited by 5 publications

(1 citation statement)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The first practical AI implementation was a handwritten digit recognizer, LeNet [95]. It led to the development of hardware for shallow neural networks, for example, Intel ETANN [96,97], SYNAPSE-1 [98], and ANNA [99]. The AI field flourished under the banner of machine learning (ML), and many self -learning algorithms were proposed to classify images, categorize text, and recognize hand-written characters [100][101][102].…”

Section: Confluence Between Artificial Intelligence and Microprocessorsmentioning

confidence: 99%

Advancements in Microprocessor Architecture for Ubiquitous AI—An Overview on History, Evolution, and Upcoming Challenges in AI Implementation

2021

View full text Add to dashboard Cite

Artificial intelligence (AI) has successfully made its way into contemporary industrial sectors such as automobiles, defense, industrial automation 4.0, healthcare technologies, agriculture, and many other domains because of its ability to act autonomously without continuous human interventions. However, this capability requires processing huge amounts of learning data to extract useful information in real time. The buzz around AI is not new, as this term has been widely known for the past half century. In the 1960s, scientists began to think about machines acting more like humans, which resulted in the development of the first natural language processing computers. It laid the foundation of AI, but there were only a handful of applications until the 1990s due to limitations in processing speed, memory, and computational power available. Since the 1990s, advancements in computer architecture and memory organization have enabled microprocessors to deliver much higher performance. Simultaneously, improvements in the understanding and mathematical representation of AI gave birth to its subset, referred to as machine learning (ML). ML includes different algorithms for independent learning, and the most promising ones are based on brain-inspired techniques classified as artificial neural networks (ANNs). ANNs have subsequently evolved to have deeper and larger structures and are often characterized as deep neural networks (DNN) and convolution neural networks (CNN). In tandem with the emergence of multicore processors, ML techniques started to be embedded in a range of scenarios and applications. Recently, application-specific instruction-set architecture for AI applications has also been supported in different microprocessors. Thus, continuous improvement in microprocessor capabilities has reached a stage where it is now possible to implement complex real-time intelligent applications like computer vision, object identification, speech recognition, data security, spectrum sensing, etc. This paper presents an overview on the evolution of AI and how the increasing capabilities of microprocessors have fueled the adoption of AI in a plethora of application domains. The paper also discusses the upcoming trends in microprocessor architectures and how they will further propel the assimilation of AI in our daily lives.

show abstract

Section: Confluence Between Artificial Intelligence and Microprocessorsmentioning

confidence: 99%

Advancements in Microprocessor Architecture for Ubiquitous AI—An Overview on History, Evolution, and Upcoming Challenges in AI Implementation

2021

View full text Add to dashboard Cite

show abstract

Exploiting domain-specific properties: compiling parallel dynamic neural network algorithms into efficient code

Prechelt¹

1999

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

AbstractÐDomain-specific constraints can be exploited to implement compiler optimizations that are not otherwise feasible. Compilers for neural network learning algorithms can achieve near-optimal colocality of data and processes and near-optimal balancing of load over processors, even for dynamically irregular problems. This is impossible for general programs, but restricting programs to the neural algorithm domain allows for the exploitation of domain-specific properties. The operations performed by neural algorithms are broadcasts, reductions, and object-local operations only; the load distribution is regular with respect to the (perhaps irregular) network topology; changes of network topology occur only from time to time. A language, compilation techniques, and a compiler implementation on the MasPar MP-1 are described and quantitative results for the effects of various optimizations used in the compiler are shown. Conservative experiments with weight pruning algorithms yield performance improvements of 27 percent due to load balancing and 195 percent improvement is achieved due to data locality, both compared to unoptimized versions. Two other optimizationsÐconnection allocation and selecting the number of replicatesÐspeed programs up by about 50 percent and 100 percent, respectively. This work can be viewed as a case study in exploiting domain-specific information; some of the principles presented here may apply to other domains as well.

show abstract

VLSI hardware for example-based learning

Lipman

Yang

1997

IEEE Trans. VLSI Syst.

View full text Add to dashboard Cite

Example-based learning, as performed by neural networks and other approximation and classification techniques, is both computationally intensive and U 0 intensive, typically involving the optimization of hundreds or thousands of parameters during repeated network evaluations over a database of example vectors. Although there is currently no dominant approach or technique among the various neural networks and learning algorithms, the basic functionality of most neural networks can be conceptually realized as a multidimensional look-up table. While multidimensional look-up tables are clearly impractical due to the exponential memory requirements, we are pursuing an approach using interpolation based only on the sparse data provided by an initial example database. In particular, we have designed prototype VLSI components for searching multidimensional example databases for the k closest examples to an input query as determined by a programmable metric using a massively parallel search. This nearest-neighbor approach can be used directly for classification, or in conjunction with any number of neural network algorithms that exploit local fitting. The hardware removes the U 0 bottleneck from the learning task by supplying a reduced set of examples for localized training or classification. Though nearest-neighbor retrieval algorithms have efficient software implementations for low-dimensional databases, exhaustive searching is the only effective approach for handling high-dimensional data. The parallel VLSI hardware we have designed can accelerate the exhaustive search by three orders of magnitude. We believe this special purpose VLSI will have direct application in systems requiring learning functionality and in accelerating learning applications on large, high-dimensional databases.

show abstract

SYNAPSE-1: a high-speed general purpose parallel neurocomputer system

Cited by 5 publications

References 3 publications

Advancements in Microprocessor Architecture for Ubiquitous AI—An Overview on History, Evolution, and Upcoming Challenges in AI Implementation

Advancements in Microprocessor Architecture for Ubiquitous AI—An Overview on History, Evolution, and Upcoming Challenges in AI Implementation

Exploiting domain-specific properties: compiling parallel dynamic neural network algorithms into efficient code

VLSI hardware for example-based learning

Contact Info

Product

Resources

About