Challenges and Trends of Nonvolatile In-Memory-Computation Circuits for AI Edge Devices

Hung, Je-Min; Jhang, Chuan-Jia; Wu, Ping-Chun; Chiu, Yen-Cheng; Chang, Meng‐Fan

doi:10.1109/ojsscs.2021.3123287

Cited by 33 publications

(18 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DACs are used to generate input voltage signals with different amplitudes according to the vector. In particular, other input methodologies have also been proposed to encode the input vectors into binary input pulse sequences or width modulated pulses according to the input precision [28]. ADCs are utilized to sample the output signals and generate the results of MVM.…”

Section: Memristive In-memory Computingmentioning

confidence: 99%

Toward memristive in-memory computing: principles and applications

Bao

Zhou

et al. 2022

Front. Optoelectron.

View full text Add to dashboard Cite

With the rapid growth of computer science and big data, the traditional von Neumann architecture suffers the aggravating data communication costs due to the separated structure of the processing units and memories. Memristive in-memory computing paradigm is considered as a prominent candidate to address these issues, and plentiful applications have been demonstrated and verified. These applications can be broadly categorized into two major types: soft computing that can tolerant uncertain and imprecise results, and hard computing that emphasizes explicit and precise numerical results for each task, leading to different requirements on the computational accuracies and the corresponding hardware solutions. In this review, we conduct a thorough survey of the recent advances of memristive in-memory computing applications, both on the soft computing type that focuses on artificial neural networks and other machine learning algorithms, and the hard computing type that includes scientific computing and digital image processing. At the end of the review, we discuss the remaining challenges and future opportunities of memristive in-memory computing in the incoming Artificial Intelligence of Things era. Graphical Abstract

show abstract

Section: Memristive In-memory Computingmentioning

confidence: 99%

Toward memristive in-memory computing: principles and applications

Bao

Zhou

et al. 2022

Front. Optoelectron.

View full text Add to dashboard Cite

show abstract

“…ADC is typically a dominant source of power consumption in CIM designs that exploit VMM operations [31]- [33]. Since the ADC power increases with ADC resolution, a general trend of RRAM-based CIM designs is to explore low ADC resolution operations [34]. This approach is suitable for neural networks applications since the networks offer inherent error robustness.…”

Section: Latency and Energy Analysismentioning

confidence: 99%

RM-NTT: An RRAM-Based Compute-in-Memory Number Theoretic Transform Accelerator

Park

Wang

Yoo

et al. 2022

IEEE J. Explor. Solid-State Comput. Devices Circuits

View full text Add to dashboard Cite

As more cloud computing resources are used for machine learning training and inference processes, privacy-preserving techniques that protect data from revealing at the cloud platforms attract increasing interest. Homomorphic encryption (HE) is one of the most promising techniques that enable privacy-preserving machine learning because HE allows data to be evaluated under encrypted forms. However, deep neural network (DNN) implementations using HE are orders of magnitude slower than plaintext implementations. The use of very long polynomials and associated number theoretic transform (NTT) operations for polynomial multiplications are the main bottlenecks of HE implementation for practical uses. This paper introduces RM-NTT, a resistive random-access memory (RRAM) based compute-in-memory (CIM) system to accelerate NTT and inverse NTT (INTT) operations. Instead of running FFT-like algorithms, RM-NTT uses a vector-matrix multiplication (VMM) approach to achieve maximal parallelism during NTT and INTT operations. To improve efficiency, RM-NTT stores modified forms of the twiddle factors in the RRAM arrays to process NTT/INTT in the same RRAM array and employs a Montgomery reduction algorithm to convert the VMM results. The proposed optimization methods allow RM-NTT to significantly reduce NTT operation latency compared with other NTT accelerators, including both CIM and non-CIM based designs. Effects of different RM-NTT design parameters and device non-idealities are also discussed.

show abstract

“…It is well-known that this extremely frequent data communication consumes very high power, which is a challenge in energy-efficient edge computing systems. Processing-in-memory (PIM) architectures have been reported to overcome the above bottleneck called memory wall [1]- [11]. In PIM architectures, each processing element has a computing circuit and a memory, reducing the frequency of the data transfer from/to external memory.…”

Section: Introductionmentioning

confidence: 99%

An Overview of Processing-in-Memory Circuits for Artificial Intelligence and Machine Learning

Kim

Xie

Chen

et al. 2022

IEEE J. Emerg. Sel. Topics Circuits Syst.

View full text Add to dashboard Cite

Artificial intelligence (AI) and machine learning (ML) are revolutionizing many fields of study, such as visual recognition, natural language processing, autonomous vehicles, and prediction. Traditional von-Neumann computing architecture with separated processing elements and memory devices have been improving their computing performances rapidly with the scaling of process technology. However, in the era of AI and ML, data transfer between memory devices and processing elements becomes the bottleneck of the system. To address this data movement issue, memory-centric computing takes an approach of merging the memory devices with processing elements so that computations can be done in the same location without moving any data. Processing-In-Memory (PIM) has attracted research community's attention because it can improve the energy efficiency of memory-centric computing systems substantially by minimizing the data movement. Even though the benefits of PIM are well accepted, its limitations and challenges have not been investigated thoroughly. This paper presents a comprehensive investigation of state-of-the-art PIM research works based on various memory device types, such as static-random-access-memory (SRAM), dynamic-random-access-memory (DRAM), and resistive memory (ReRAM). We will present the overview of PIM designs in each memory type, covering from bit cells, circuits, and architecture. Then, a new software stack standard and its challenges for incorporating PIM with the conventional computing architecture will be discussed. Finally, we will discuss various future research directions in PIM for further reducing the data conversion overhead, improving test accuracy, and minimizing intra-memory data movement.

show abstract

Challenges and Trends of Nonvolatile In-Memory-Computation Circuits for AI Edge Devices

Cited by 33 publications

References 61 publications

Toward memristive in-memory computing: principles and applications

Toward memristive in-memory computing: principles and applications

RM-NTT: An RRAM-Based Compute-in-Memory Number Theoretic Transform Accelerator

An Overview of Processing-in-Memory Circuits for Artificial Intelligence and Machine Learning

Contact Info

Product

Resources

About