Understanding Acceleration Opportunities at Hyperscale

Abstract

Modern web services run across hundreds of thousands of servers in a data center, i.e., at hyperscale. With the end of Moore's Law and Dennard scaling, successive server generations running these web services exhibit diminishing performance returns, resulting in architects adopting hardware customization. An important question arises: Which web service software operations are worth building custom hardware for? To answer this question, we comprehensively analyze seven important Facebook production services and identify key acceleration opportunities. We then develop an open-source analytical model, Accelerometer, to help make well-informed hardware decisions for the acceleration opportunities we identify.

Don't be a blockhead

“…This apparent contradiction in system design philosophies calls for academic scrutiny. Similar to Facebook's Accelerometer for computation offloading [48], we envision research on how to decide which parts of the hardware stack should be responsible for which functionality on the I/O control and datapaths.…”

We need kernel interposition over the network dataplane

“…In this section, we consider Alice, a system administrator managing a server with users Bob and Charlie. 1 Alice is in a bare-metal deployment environment (e.g., deployments at Facebook [43,48], containers at Google [5,54], etc. [55]).…”

Understanding Data Storage and Ingestion for Large-Scale Deep Recommendation Model Training

“…For example, we are experimenting with more efficient authentication tokens, which has shown a 12 percent increase in DPP Client throughput. Longer term, we are investigating opportunities to accelerate "datacenter tax" operations [31,45]. Note that the use of accelerators will not eliminate the need to right-size data preprocessing for each model using our disaggregated architecture.…”