High-Performance Deep-Learning Coprocessor Integrated into x86 SoC with Server-Class CPUs Industrial Product

Henry, Glenn; Palangpour, Parviz; Thomson, Michael; Gardner, Josh; Arden, Bryce; Donahue, Jim; Houck, Kimble; Johnson, Jonathan; O’Brien, Kyle; Petersen, Scott R.; Seroussi, Benjamin; Walker, Tyler

doi:10.1109/isca45697.2020.00013

Cited by 12 publications

(6 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2) Target Architecture: CoSA targets spatial architectures with an array of processing elements (PEs) connected via an on-chip network and with multiple levels of memory hierarchy, a commonly adopted architecture template in today's DNN accelerator designs [18], [19], [28], [29], [36], [44], [54], [55], [60], [69], [71].…”

Section: A Cosa Overviewmentioning

confidence: 99%

“…DNN-based approaches have been applied to computer vision [34], [43], [57], machine translation [64], [68], audio synthesis [66], recommendation models [31], [46], autonomous driving [11] and many other fields. Motivated by the high computational requirements of DNNs, there have been exciting developments in both research and commercial spaces in building specialized DNN accelerators for both edge [1], [16], [17], [26], [50], [61], [63], [72] and cloud applications [5], [19], [27], [36], [39], [69].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

CoSA: Scheduling by Constrained Optimization for Spatial Accelerators

Huang¹,

Kang²,

Dinh³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent advances in Deep Neural Networks (DNNs) have led to active development of specialized DNN accelerators, many of which feature a large number of processing elements laid out spatially, together with a multi-level memory hierarchy and flexible interconnect. While DNN accelerators can take advantage of data reuse and achieve high peak throughput, they also expose a large number of runtime parameters to the programmers who need to explicitly manage how computation is scheduled both spatially and temporally. In fact, different scheduling choices can lead to wide variations in performance and efficiency, motivating the need for a fast and efficient search strategy to navigate the vast scheduling space.To address this challenge, we present CoSA, a constrainedoptimization-based approach for scheduling DNN accelerators. As opposed to existing approaches that either rely on designers' heuristics or iterative methods to navigate the search space, CoSA expresses scheduling decisions as a constrained-optimization problem that can be deterministically solved using mathematical optimization techniques. Specifically, CoSA leverages the regularities in DNN operators and hardware to formulate the DNN scheduling space into a mixed-integer programming (MIP) problem with algorithmic and architectural constraints, which can be solved to automatically generate a highly efficient schedule in one shot. We demonstrate that CoSA-generated schedules significantly outperform state-of-the-art approaches by a geometric mean of up to 2.5× across a wide range of DNN networks while improving the time-to-solution by 90×.

show abstract

Section: A Cosa Overviewmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

CoSA: Scheduling by Constrained Optimization for Spatial Accelerators

Huang¹,

Kang²,

Dinh³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Some portions of the NS-DR and NSCL are more suited for acceleration than others. The CNN-based video frame parser and the transformer-based question parser model can be accelerated with existing GPU or accelerator architectures [34]. On the other hand, the NS-DR's dynamics predictor is severely data-movement-bound due to its many sparse and small tensor operations, and also suffers from the difficulty of multiplying "tall-and-skinny" matrices.…”

Section: A Nscl and Ns-drmentioning

confidence: 99%

Neuro-Symbolic AI: An Emerging Class of AI Workloads and their Characterization

Susskind¹,

Arden²,

John³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Neuro-symbolic artificial intelligence is a novel area of AI research which seeks to combine traditional rules-based AI approaches with modern deep learning techniques. Neurosymbolic models have already demonstrated the capability to outperform state-of-the-art deep learning models in domains such as image and video reasoning. They have also been shown to obtain high accuracy with significantly less training data than traditional models. Due to the recency of the field's emergence and relative sparsity of published results, the performance characteristics of these models are not well understood. In this paper, we describe and analyze the performance characteristics of three recent neuro-symbolic models. We find that symbolic models have less potential parallelism than traditional neural models due to complex control flow and low-operational-intensity operations, such as scalar multiplication and tensor addition. However, the neural aspect of computation dominates the symbolic part in cases where they are clearly separable. We also find that data movement poses a potential bottleneck, as it does in many ML workloads.

show abstract

“…• SambaNova has released some impressive benchmark results for their reconfigurable AI accelerator technology, but they still have not provided any details from which we can estimate peak performance or power consumption of their solutions [123]. • The Centaur Technology CNS processor [124], [125] includes eight x86 cores along with an integrated AI accelerator realized as a 4,096 byte-wide SIMD unit. The Centaur AI coprocessor (CT-AIC) will delivers peak performance of 20 TOPS with INT8 precision at 2.5 GHz and can also operate at FP16 and INT16 precisions, though at lower performance.…”

Section: A New Acceleratorsmentioning

confidence: 99%

AI Accelerator Survey and Trends

Reuther,

Michaleas,

Jones

et al. 2021

Preprint

View full text Add to dashboard Cite

Over the past several years, new machine learning accelerators were being announced and released every month for a variety of applications from speech recognition, video object detection, assisted driving, and many data center applications. This paper updates the survey of AI accelerators and processors from past two years. This paper collects and summarizes the current commercial accelerators that have been publicly announced with peak performance and power consumption numbers. The performance and power values are plotted on a scatter graph, and a number of dimensions and observations from the trends on this plot are again discussed and analyzed. This year, we also compile a list of benchmarking performance results and compute the computational efficiency with respect to peak performance.

show abstract

High-Performance Deep-Learning Coprocessor Integrated into x86 SoC with Server-Class CPUs Industrial Product

Cited by 12 publications

References 14 publications

CoSA: Scheduling by Constrained Optimization for Spatial Accelerators

CoSA: Scheduling by Constrained Optimization for Spatial Accelerators

Neuro-Symbolic AI: An Emerging Class of AI Workloads and their Characterization

AI Accelerator Survey and Trends

Contact Info

Product

Resources

About