FARM: A Flexible Accelerator for Recurrent and Memory Augmented Neural Networks

Challapalle, Nagadastagiri; Rampalli, Sahithi; Jao, Nicholas; Ramanathan, Akshay Krishna; Sampson, John J.; Narayanan, Vijaykrishnan

doi:10.1007/s11265-020-01555-w

Cited by 7 publications

(15 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The history-based read weighting is computed in three steps: 1) the write weighting 𝑤 𝑤 is first expanded to an 𝑁 × 𝑁 matrix to derive linkage matrix 𝐿. The linkage tracks the order in which memory locations are written to; 2) the precedence vector 𝑝 is updated to track the degree each memory entry is most [4] and tiled architecture [33] for accelerating MANN's memory unit. recently written to; and 2) a forward and backward pass is used to merge the read weighting 𝑤 𝑟 from the previous time step with the linkage matrix 𝐿, as well as the content-based read weighting 𝑟 𝑢 to update the read weighting 𝑤 𝑟 .…”

Section: Softmentioning

confidence: 99%

“…MANNA's distributed architecture provides more memory bandwidth and compute parallelism, but its H-tree NoC still incurs a traffic bottleneck when running DNC's history-based attention mechanisms. DNC accelerators have been developed recently [4,30]. In [4], the efficiency mainly comes from analog-based processing elements such as analog-to-digital converters (ADCs), which are more sensitive to variations and noise, and less portable between process technologies.…”

Section: Introductionmentioning

confidence: 99%

“…DNC accelerators have been developed recently [4,30]. In [4], the efficiency mainly comes from analog-based processing elements such as analog-to-digital converters (ADCs), which are more sensitive to variations and noise, and less portable between process technologies. These designs all followed a centralized architecture for memory access and compute, which could lead to poor scalability when memory size increases.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

HiMA: A Fast and Scalable History-based Memory Access Engine for Differentiable Neural Computer

Tao

Zhang

2021

MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture

View full text Add to dashboard Cite

Memory-augmented neural networks (MANNs) provide better inference performance in many tasks with the help of an external memory. The recently developed differentiable neural computer (DNC) is a MANN that has been shown to outperform in representing complicated data structures and learning long-term dependencies. DNC's higher performance is derived from new history-based attention mechanisms in addition to the previously used contentbased attention mechanisms. History-based mechanisms require a variety of new compute primitives and state memories, which are not supported by existing neural network (NN) or MANN accelerators. We present HiMA, a tiled, history-based memory access engine with distributed memories in tiles. HiMA incorporates a multi-mode network-on-chip (NoC) to reduce the communication latency and improve scalability. An optimal submatrix-wise memory partition strategy is applied to reduce the amount of NoC traffic; and a two-stage usage sort method leverages distributed tiles to improve computation speed. To make HiMA fundamentally scalable, we create a distributed version of DNC called DNC-D to allow almost all memory operations to be applied to local memories with trainable weighted summation to produce the global memory output. Two approximation techniques, usage skimming and softmax approximation, are proposed to further enhance hardware efficiency. HiMA prototypes are created in RTL and synthesized in a 40nm technology. By simulations, HiMA running DNC and DNC-D demonstrates 6.47× and 39.1× higher speed, 22.8× and 164.3× better area efficiency, and 6.1× and 61.2× better energy efficiency over the state-of-the-art MANN accelerator. Compared to an Nvidia 3080Ti GPU, HiMA demonstrates speedup by up to 437× and 2,646× when running DNC and DNC-D, respectively. CCS CONCEPTS• Computer systems organization → Neural networks; Data flow architectures; Special purpose systems.

show abstract

Section: Softmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

HiMA: A Fast and Scalable History-based Memory Access Engine for Differentiable Neural Computer

Tao

Zhang

2021

MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture

View full text Add to dashboard Cite

show abstract

“…In this manner, MCA is able to perform the matrix multiplication operation in just one step. Resistive memory in its memristive crossbar array (MCA) configuration can significantly accelerate matrix multiply operations commonly found in neural network, graph, and image processing workloads [19,[82][83][84][85][86][87][88][89]. Figure 5a shows a mathematical abstraction of one such operation for a single-layer perceptron implementing binary classification, where x are the inputs to the perceptron, w are the weights to the perceptron, sgn is the signum function, and y is the output of the perceptron [90].…”

Section: Memory Typesmentioning

confidence: 99%

“…In contrast, DRAM requires multiple loads/stores and computation steps to perform the same operation. This advantage is exploited by management policies which map matrix vector multiplication to ReRAM crossbar arrays [19,[82][83][84][85][86][87][88][89].…”

Section: Energy Efficiencymentioning

confidence: 99%

A Survey of Resource Management for Processing-In-Memory and Near-Memory Processing Architectures

Khan

Pasricha

Kim

2020

JLPEA

View full text Add to dashboard Cite

Due to the amount of data involved in emerging deep learning and big data applications, operations related to data movement have quickly become a bottleneck. Data-centric computing (DCC), as enabled by processing-in-memory (PIM) and near-memory processing (NMP) paradigms, aims to accelerate these types of applications by moving the computation closer to the data. Over the past few years, researchers have proposed various memory architectures that enable DCC systems, such as logic layers in 3D-stacked memories or charge-sharing-based bitwise operations in dynamic random-access memory (DRAM). However, application-specific memory access patterns, power and thermal concerns, memory technology limitations, and inconsistent performance gains complicate the offloading of computation in DCC systems. Therefore, designing intelligent resource management techniques for computation offloading is vital for leveraging the potential offered by this new paradigm. In this article, we survey the major trends in managing PIM and NMP-based DCC systems and provide a review of the landscape of resource management techniques employed by system designers for such systems. Additionally, we discuss the future challenges and opportunities in DCC management.

show abstract

Advances in Data Preprocessing for Biomedical Data Fusion: An Overview of the Methods, Challenges, and Prospects

et al. 2021

View full text Add to dashboard Cite

FARM: A Flexible Accelerator for Recurrent and Memory Augmented Neural Networks

Cited by 7 publications

References 22 publications

HiMA: A Fast and Scalable History-based Memory Access Engine for Differentiable Neural Computer

HiMA: A Fast and Scalable History-based Memory Access Engine for Differentiable Neural Computer

A Survey of Resource Management for Processing-In-Memory and Near-Memory Processing Architectures

Advances in Data Preprocessing for Biomedical Data Fusion: An Overview of the Methods, Challenges, and Prospects

Contact Info

Product

Resources

About