CHIMERA: A 0.92 TOPS, 2.2 TOPS/W Edge AI Accelerator with 2 MByte On-Chip Foundry Resistive RAM for Efficient Training and Inference

Giordano, Massimo; Prabhu, Kartik; Koul, Kalhan; Radway, Robert M.; Gural, Albert; Doshi, Rohan; Khan, Zainab F.; Kustin, John W.; Liu, Timothy; Lopes, Gregorio B.; Turbiner, Victor; Khwa, Win-San; Chih, Yu-Der; Chang, Meng‐Fan; Lallement, Guénolé; Murmann, Boris; Mitra, Subhasish; Raina, Priyanka

doi:10.23919/vlsicircuits52068.2021.9492347

Cited by 37 publications

(19 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The overhead of inserting our annotations for dFC can be small compared to what designers already insert to optimize the design. For ISmartDNN, for example, the total number of annotations is 304, which is 2.8% of the (5) 3 / 3 / 0 mean128 [33] (202k) (5) timeout 5 / 5 / 0 mean64 [33] (104k) (5) timeout 3 / 3 / 0 mean32 [33] (54k) (5) 1 / 1 / 0 dnn [57] (2M) (11) timeout 5 / 5 / 0 keypair [58] (>200M) (1) timeout 21 / 21 / 0 gsm [59] (8.8k) (1) timeout 7 / 7 / 0 nv_large [12] (16M) (23) timeout No RB bugs expected nv_small [12] (1M) (23) timeout HLSCNN [60] (323k) ( I. A-QED 2 RB checks are performed on all sub-accelerators regardless of batch size, so P is omitted compared to Table I.…”

Section: Resultsmentioning

confidence: 99%

Scaling Up Hardware Accelerator Verification using A-QED with Functional Decomposition

Chattopadhyay,

Lonsing,

Piccolboni

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Hardware accelerators (HAs) are essential building blocks for fast and energy-efficient computing systems. Accelerator Quick Error Detection (A-QED) is a recent formal technique which uses Bounded Model Checking for pre-silicon verification of HAs. A-QED checks an HA for self-consistency, i.e., whether identical inputs within a sequence of operations always produce the same output. Under modest assumptions, A-QED is both sound and complete. However, as is well-known, large design sizes significantly limit the scalability of formal verification, including A-QED. We overcome this scalability challenge through a new decomposition technique for A-QED, called A-QED with Decomposition (A-QED 2 ). A-QED 2 systematically decomposes an HA into smaller, functional sub-modules, called sub-accelerators, which are then verified independently using A-QED. We prove completeness of A-QED 2 ; in particular, if the full HA under verification contains a bug, then A-QED 2 ensures detection of that bug during A-QED verification of the corresponding subaccelerators. Results on over 100 (buggy) versions of a wide variety of HAs with millions of logic gates demonstrate the effectiveness and practicality of A-QED 2 . This article will appear in the proceedings of Formal Methods in Computer-Aided Design (FMCAD 2021).

show abstract

Section: Resultsmentioning

confidence: 99%

Scaling Up Hardware Accelerator Verification using A-QED with Functional Decomposition

Chattopadhyay,

Lonsing,

Piccolboni

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The latency and throughput of EfficientNetV2-S are the best with 1.70 ms and 588.2 Inferences/s, respectively. The energy efficiency of 128.1 Inferences/s/W is comparable to the onchip resistive RAM implementation of 132.3 Inferences/s/W [38], whereas throughput is 202× higher.…”

Section: Implementation and Measurementsmentioning

confidence: 95%

A 64-TOPS Energy-Efficient Tensor Accelerator in 14nm With Reconfigurable Fetch Network and Processing Fusion for Maximal Data Reuse

Lee¹,

Kim²,

Yeon³

et al. 2022

IEEE Open J. Solid-State Circuits Soc.

View full text Add to dashboard Cite

For energy-efficient accelerators in data centers that leverage advances in the performance and energy efficiency of recent algorithms, flexible architectures are critical to support state-of-the-art algorithms for various deep learning tasks. Due to the matrix multiplication units at the core of tensor operations, most recent programmable architectures lack flexibility for layers with diminished dimensions, especially for inferences where a large batch axis is rarely allowed. In addition, exploiting the data reuse inherent within tensor operations for computing a single matrix multiplication is challenging. In this work, an extension of a vector processor in 14 nm is proposed, which is customized to tensor operations. The flexible architecture enables a tensorized loop to support various data layouts and different shapes and sizes of tensor operations. It also exploits all possible data reuse, including input, weight, and output. Based on the tensorized loop, fetch and reduction networks, which unicast or multicast with the ordering of both input data and processing data, can be simplified using a circuit-switching-like network with configured topology and flow control for each tensor operation. Two processing elements can be fused to optimize latency for a large model or can operate individually for throughput. As a result, various state-of-the-art models can be processed efficiently with straightforward compiler optimization, and the highest energy efficiency of 13.4 Inferences/s/W on EfficientNetV2-S is demonstrated.

show abstract

“…Better algorithms [24] [2] are designed to more efficiently use resources, improve model performance, and optimize the deployment in a real-world environment. Also, ultra-low-power AI chips [10] and accelerators [15] have been proposed to support always-on ML capability for an extended period by a battery. However, a joint design of hardware and algorithm [30] [13] is required to squeeze the performance since TinyML delivers ML solutions to constrained devices with limited resources.…”

Section: Related Workmentioning

confidence: 99%

How to Manage Tiny Machine Learning at Scale: An Industrial Perspective

Ren¹,

Anicic²,

Runkler³

2022

Preprint

View full text Add to dashboard Cite

Tiny machine learning (TinyML) has gained widespread popularity where machine learning (ML) is democratized on ubiquitous microcontrollers, processing sensor data everywhere in real-time. To manage TinyML in the industry, where mass deployment happens, we consider the hardware and software constraints, ranging from available onboard sensors and memory size to ML-model architectures and runtime platforms. However, Internet of Things (IoT) devices are typically tailored to specific tasks and are subject to heterogeneity and limited resources. Moreover, TinyML models have been developed with different structures and are often distributed without a clear understanding of their working principles, leading to a fragmented ecosystem. Considering these challenges, we propose a framework using Semantic Web technologies to enable the joint management of TinyML models and IoT devices at scale, from modeling information to discovering possible combinations and benchmarking, and eventually facilitate TinyML component exchange and reuse. We present an ontology (semantic schema) for neural network models aligned with the World Wide Web Consortium (W3C) Thing Description, which semantically describes IoT devices. Furthermore, a Knowledge Graph of 23 publicly available ML models and six IoT devices were used to demonstrate our concept in three case studies, and we shared the code and examples to enhance reproducibility: Github

show abstract

CHIMERA: A 0.92 TOPS, 2.2 TOPS/W Edge AI Accelerator with 2 MByte On-Chip Foundry Resistive RAM for Efficient Training and Inference

Cited by 37 publications

References 0 publications

Scaling Up Hardware Accelerator Verification using A-QED with Functional Decomposition

Scaling Up Hardware Accelerator Verification using A-QED with Functional Decomposition

A 64-TOPS Energy-Efficient Tensor Accelerator in 14nm With Reconfigurable Fetch Network and Processing Fusion for Maximal Data Reuse

How to Manage Tiny Machine Learning at Scale: An Industrial Perspective

Contact Info

Product

Resources

About