Fused-layer CNN accelerators

Alwani, Manoj; Chen, Han; Ferdman, Michael; Milder, Peter

doi:10.1109/micro.2016.7783725

Cited by 489 publications

(305 citation statements)

References 13 publications

Supporting

Mentioning

302

Contrasting

Unclassified

Order By: Relevance

“…With much larger storage on the order of a few megabytes, additional dataflows can be considered. For example, Fused-Layer looks at dataflow optimizations across layers [96]. …”

Section: ) Energy Comparison Of Different Dataflowsmentioning

confidence: 99%

Efficient Processing of Deep Neural Networks: A Tutorial and Survey

et al. 2017

View full text Add to dashboard Cite

Abstract-Deep neural networks (DNNs) are currently widely used for many artificial intelligence (AI) applications including computer vision, speech recognition, and robotics. While DNNs deliver state-of-the-art accuracy on many AI tasks, it comes at the cost of high computational complexity. Accordingly, techniques that enable efficient processing of DNNs to improve energy efficiency and throughput without sacrificing application accuracy or increasing hardware cost are critical to the wide deployment of DNNs in AI systems.This article aims to provide a comprehensive tutorial and survey about the recent advances towards the goal of enabling efficient processing of DNNs. Specifically, it will provide an overview of DNNs, discuss various hardware platforms and architectures that support DNNs, and highlight key trends in reducing the computation cost of DNNs either solely via hardware design changes or via joint hardware design and DNN algorithm changes. It will also summarize various development resources that enable researchers and practitioners to quickly get started in this field, and highlight important benchmarking metrics and design considerations that should be used for evaluating the rapidly growing number of DNN hardware designs, optionally including algorithmic co-designs, being proposed in academia and industry.The reader will take away the following concepts from this article: understand the key design considerations for DNNs; be able to evaluate different DNN hardware implementations with benchmarks and comparison metrics; understand the trade-offs between various hardware architectures and platforms; be able to evaluate the utility of various DNN design techniques for efficient processing; and understand recent implementation trends and opportunities.

show abstract

“…With much larger storage on the order of a few megabytes, additional dataflows can be considered. For example, Fused-Layer looks at dataflow optimizations across layers [96]. …”

Section: ) Energy Comparison Of Different Dataflowsmentioning

confidence: 99%

Efficient Processing of Deep Neural Networks: A Tutorial and Survey

et al. 2017

View full text Add to dashboard Cite

show abstract

“…1,2 Accenting the fairness of the comparison, we note that the Single-CLP and Multi-CLP designs have the same arithmetic unit cost, which the Multi-CLP design spreads among several CLPs. Recall that a CLP requires T n × T m multipliers and adders.…”

Section: Detailed Comparison: Single-vs Multi-clpmentioning

confidence: 99%

“…[21] uses per-layer data quantization and matrix-decomposition, whereas [14] uses perlayer numerical precision reduction. [2] uses a fused-layer technique to reduce bandwidth use of convolutional layers. [25] optimizes batch sizes to reduce off-chip data transfer.…”

Section: Related Workmentioning

confidence: 99%

Maximizing CNN Accelerator Efficiency Through Resource Partitioning

ShenYongming

FerdmanMichael

MilderPeter

2017

SIGARCH Comput. Archit. News

Self Cite

132

149

View full text Add to dashboard Cite

Convolutional neural networks (CNNs) are revolutionizing machine learning, but they present significant computational challenges. Recently, many FPGA-based accelerators have been proposed to improve the performance and efficiency of CNNs. Current approaches construct a single processor that computes the CNN layers one at a time; the processor is optimized to maximize the throughput at which the collection of layers is computed. However, this approach leads to inefficient designs because the same processor structure is used to compute CNN layers of radically varying dimensions.We present a new CNN accelerator paradigm and an accompanying automated design methodology that partitions the available FPGA resources into multiple processors, each of which is tailored for a different subset of the CNN convolutional layers. Using the same FPGA resources as a single large processor, multiple smaller specialized processors increase computational efficiency and lead to a higher overall throughput. Our design methodology achieves 3.8x higher throughput than the state-of-the-art approach on evaluating the popular AlexNet CNN on a Xilinx Virtex-7 FPGA. For the more recent SqueezeNet and GoogLeNet, the speedups are 2.2x and 2.0x.

show abstract

“…In the aforementioned approaches, some studies ( [7], [11], [19], [26]- [29]) have considered communication optimization, while the others mainly focus on the computational components. Besides the three categories, there are other communication optimization approaches for CNN accelerators (e.g., the fused-layer approach [35] that optimizes data movement between convolutional layers). Currently, no study has comprehensively analyzed the lower bound of communication in CNN accelerators.…”

Section: B Related Workmentioning

confidence: 99%

Communication Lower Bound in Convolution Accelerators

Chen

Han

Wang

2020

2020 IEEE International Symposium on High Performance Computer Architecture (HPCA)

View full text Add to dashboard Cite

In current convolutional neural network (CNN) accelerators, communication (i.e., memory access) dominates the energy consumption. This work provides comprehensive analysis and methodologies to minimize the communication for CNN accelerators. For the off-chip communication, we derive the theoretical lower bound for any convolutional layer and propose a dataflow to reach the lower bound. This fundamental problem has never been solved by prior studies. The on-chip communication is minimized based on an elaborate workload and storage mapping scheme. We in addition design a communication-optimal CNN accelerator architecture. Evaluations based on the 65nm technology demonstrate that the proposed architecture nearly reaches the theoretical minimum communication in a threelevel memory hierarchy and it is computation dominant. The gap between the energy efficiency of our accelerator and the theoretical best value is only 37-87%.

show abstract

Fused-layer CNN accelerators

Cited by 489 publications

References 13 publications

Efficient Processing of Deep Neural Networks: A Tutorial and Survey

Efficient Processing of Deep Neural Networks: A Tutorial and Survey

Maximizing CNN Accelerator Efficiency Through Resource Partitioning

Communication Lower Bound in Convolution Accelerators

Contact Info

Product

Resources

About