Demystifying the MLPerf Training Benchmark Suite

Verma, Snehil; Wu, Qinzhe; Hanindhito, Bagus; Jha, Gunjan; John, Eugene; Radhakrishnan, Ramesh; John, Lizy K.

doi:10.1109/ispass48437.2020.00013

Cited by 6 publications

(1 citation statement)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Training [57]. Cross-layer approaches related to our work include high- [52] and low- [29] level code generation techniques, and also memory management [23] and memory partitioning techniques [25], [35], [53].…”

Section: Verma Et Al Present a Workload Characterization Of Mlperfmentioning

confidence: 99%

Understanding the Limits of Conventional Hardware Architectures for Deep-Learning

Davies¹,

Labiosa²,

Sankaralingam³

2021

Preprint

View full text Add to dashboard Cite

Deep learning and hardware for it has garnered immense academic and industry interest in the past 5 yearsincluding almost 100 startups, more than $5B of VC investment -and a re-relevance of the role of architecture. However, the state-of-art remains NVIDIA's TensorCore-based systems that provide i) top-of-line performance, ii) turnkey software stack, and iii) coverage across a wide-spectrum of DL network styles (DL-architecture in AI parlance). Other academic and industry efforts have included novel approaches like spatial dataflow, CGRAs, systolic arrays, blended FPGA LUTs with fixed function units and more. These have all necessitated their own innovations in architecture, compiler, and software stack integration. However, none of these have yet satisfied all the 3 metrics that NVIDIA's TensorCore and software stack provides, and generally seem to perform worse. In this paper, we systematically investigate the behavior of DL workloads and imputed needs on hardware/compiler/software. We show that SIMD/shortvector, caching, and synchronization in a fairly well-understood multicore chip organization we call UPCYCLE can achieve dayzero software maturity, and provide big integer factor speedups over the state-of-art NVIDIA solutions. Compared to an A100, UPCYCLE at small-batch size is geo-mean 7.7X faster for inference, geo-mean 11.7X faster at training, while consuming only half the power. Second, the UPCYCLE architecture requires no new compiler or software stack innovation. Third, it provides full DL-architecture coverage, and can be instantiated to provide training-optimized, inference-optimized, or balanced training and inference systems. Overall, this paper motivates the treatment of software maturity as a first class design constraint in developing new architectures for DL.

show abstract