DeepRecSys: A System for Optimizing End-To-End At-scale Neural Recommendation Inference

Gupta, Udit; Hsia, Samuel; Saraph, Vikram; Wang, Xiaodong; Reagen, Brandon; Wei, Gu-Yeon; Lee, Hsien-Hsin S.; Brooks, David; Wu, Carole-Jean

doi:10.48550/arxiv.2001.02772

Cited by 10 publications

(26 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared with the first configuration, the second has dual AMD Radeon Vega GPUs, providing 4×, 8×, and 16× more flops, memory bandwidth, and capacity, respectively. It represents a data-center-scale server [76], [77], yielding a 2.6× greater manufacturing carbon footprint.…”

Section: Addressing Carbon Footprint Of Systemsmentioning

confidence: 99%

“…Optimizing for capex-related emissions requires reducing hardware resources. Recently proposed schedulers optimize infrastructure efficiency in terms of total power consumption while balancing performance for latency-critical and batchprocessing workloads [76], [77], [86]- [88]. Others have proposed novel scheduler designs to enable energy-efficient, collaborative cloud and edge execution [66], [89].…”

Section: Addressing Carbon Footprint Of Systemsmentioning

confidence: 99%

See 1 more Smart Citation

Chasing Carbon: The Elusive Environmental Footprint of Computing

Gupta

Kim

Lee

et al. 2021

2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA)

Self Cite

View full text Add to dashboard Cite

Given recent algorithm, software, and hardware innovation, computing has enabled a plethora of new applications. As computing becomes increasingly ubiquitous, however, so does its environmental impact. This paper brings the issue to the attention of computer-systems researchers. Our analysis, built on industry-reported characterization, quantifies the environmental effects of computing in terms of carbon emissions. Broadly, carbon emissions have two sources: operational energy consumption, and hardware manufacturing and infrastructure. Although carbon emissions from the former are decreasing thanks to algorithmic, software, and hardware innovations that boost performance and power efficiency, the overall carbon footprint of computer systems continues to grow. This work quantifies the carbon output of computer systems to show that most emissions related to modern mobile and data-center equipment come from hardware manufacturing and infrastructure. We therefore outline future directions for minimizing the environmental impact of computing systems.

show abstract

Section: Addressing Carbon Footprint Of Systemsmentioning

confidence: 99%

Section: Addressing Carbon Footprint Of Systemsmentioning

confidence: 99%

Chasing Carbon: The Elusive Environmental Footprint of Computing

Gupta

Kim

Lee

et al. 2021

2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA)

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the past decade, deep learning recommendation models have successfully powered various applications and products. For instance, YouTube adopts deep candidate generation and ranking models to deliver video suggestions [1]; Microsoft utilizes deep learning recommendation models for news feed [2]; Alibaba leverages recommendation models for product suggestions [3]; Facebook uses deep learning recommendation models to support personalization and ranking in widespread products [4]- [6]. Recent years, along with the emergence of big data, recommendation models are rapidly expanding their capacity, especially the industry recommendation systems.…”

Section: Introductionmentioning

confidence: 99%

Alternate Model Growth and Pruning for Efficient Training of Recommendation Systems

Du¹,

Bhushanam²,

Yu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep learning recommendation systems at scale have provided remarkable gains through increasing model capacity (i.e. wider and deeper neural networks), but it comes at significant training cost and infrastructure cost. Model pruning is an effective technique to reduce computation overhead for deep neural networks by removing redundant parameters. However, modern recommendation systems are still thirsty for model capacity due to the demand for handling big data. Thus, pruning a recommendation model at scale results in a smaller model capacity and consequently lower accuracy. To reduce computation cost without sacrificing model capacity, we propose a dynamic training scheme, namely alternate model growth and pruning, to alternatively construct and prune weights in the course of training. Our method leverages structured sparsification to reduce computational cost without hurting the model capacity at the end of offline training so that a full-size model is available in the recurring training stage to learn new data in real time. To the best of our knowledge, this is the first work to provide in-depth experiments and discussion of applying structural dynamics to recommendation systems at scale to reduce training cost. The proposed method is validated with an open-source deep learning recommendation model (DLRM) and the state-of-theart industrial-scale production models.

show abstract

“…Recommender Systems serve to personalize user experience in a variety of applications including predicting click-through rates for ranking advertisements [34], improving search results [30], suggesting friends and content on social networks [30], suggesting food on Uber Eats [53], helping users find houses on Zillow [54], helping contain information overload by suggesting relevant news articles [55], helping users find videos to watch on YouTube [43] and movies on Netflix [59], and several more real-world use cases [60]. An introduction to recommender system technology can be found in [58] and a set of best practices and examples for building recommender systems in [56].…”

Section: Introductionmentioning

confidence: 99%

“…The focus of this paper is recommendation systems that use neural networks, referred to as Deep Learning RecSys, or simply RecSys 1 . These have been recently applied to a variety of areas with success [34] [30].…”

Section: Introductionmentioning

confidence: 99%

Accelerating Recommender Systems via Hardware "scale-in"

Kumar¹,

Krishna²

2020

Preprint

View full text Add to dashboard Cite

In today's era of "scale-out", this paper makes the case that a specialized hardware architecture based on "scalein"-placing as many specialized processors as possible along with their memory systems and interconnect links within one or two boards in a rack-would offer the potential to boost large recommender system throughput by 12-62× for inference and 12-45× for training compared to the DGX-2 state-of-the-art AI platform, while minimizing the performance impact of distributing large models across multiple processors. By analyzing Facebook's representative model-Deep Learning Recommendation Model (DLRM)-from a hardware architecture perspective, we quantify the impact on throughput of hardware parameters such as memory system design, collective communications latency and bandwidth, and interconnect topology. By focusing on conditions that stress hardware, our analysis reveals limitations of existing AI accelerators and hardware platforms. * Equal contribution.1 While the term is often used to denote any recommender system, it is specifically used for Deep Learning recommenders here.

show abstract

DeepRecSys: A System for Optimizing End-To-End At-scale Neural Recommendation Inference

Cited by 10 publications

References 56 publications

Chasing Carbon: The Elusive Environmental Footprint of Computing

Chasing Carbon: The Elusive Environmental Footprint of Computing

Alternate Model Growth and Pruning for Efficient Training of Recommendation Systems

Accelerating Recommender Systems via Hardware "scale-in"

Contact Info

Product

Resources

About