Challenges and Opportunities of DNN Model Execution Caching

Gilman, Guin; Ogden, Samuel S.; Walls, Robert J.; Guo, Tian

doi:10.1145/3366622.3368147

Cited by 6 publications

(1 citation statement)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An interesting research question is then how to manage a large number of deep learning models given dynamic workload to satisfy performance objectives while lowering resource costs [91]. One promising direction is to design deep learning-specific caching algorithms to manage main memory resources and minimize the performance impact of the model cold start problem [87]. Formulating the model management problem as a caching problem allows leveraging rich literature on caching; however, questions such as how to incorporate the relatively slow PCIe transfer to GPUs and effectively use virtual GPU memory remain unsolved.…”

Section: Iot and Ai Are Becoming The Main Applicationsmentioning

confidence: 99%

On the Future of Cloud Engineering

Bermbach

Chandra

Krintz

et al. 2021

2021 IEEE International Conference on Cloud Engineering (IC2E)

Self Cite

View full text Add to dashboard Cite

Ever since the commercial offerings of the Cloud started appearing in 2006, the landscape of cloud computing has been undergoing remarkable changes with the emergence of many different types of service offerings, developer productivity enhancement tools, and new application classes as well as the manifestation of cloud functionality closer to the user at the edge. The notion of utility computing, however, has remained constant throughout its evolution, which means that cloud users always seek to save costs of leasing cloud resources while maximizing their use. On the other hand, cloud providers try to maximize their profits while assuring service-level objectives of the cloud-hosted applications and keeping operational costs low. All these outcomes require systematic and sound cloud engineering principles. The aim of this paper is to highlight the importance of cloud engineering, survey the landscape of best practices in cloud engineering and its evolution, discuss many of the existing cloud engineering advances, and identify both the inherent technical challenges and research opportunities for the future of cloud computing in general and cloud engineering in particular.

show abstract

Section: Iot and Ai Are Becoming The Main Applicationsmentioning

confidence: 99%