Modern websites increasingly rely on machine learning (ML) to improve their business efficiency. Developing and maintaining ML services incurs high costs for developers. Although serverless systems are a promising solution to reduce costs, we find that the current general purpose serverless systems cannot meet the low latency, high throughput demands of ML services.While simply łpatchingž general serverless systems does not resolve the problem completely, we propose that such a system should natively combine the features of inference with a serverless paradigm. We present INFless, the first ML domain-specific serverless platform. It provides a unified, heterogeneous resource abstraction between CPU and accelerators, and achieves high throughput using built-in batching and non-uniform scaling mechanisms. It also supports low latency through coordinated management of batch queuing time, execution time and coldstart rate. We evaluate INFless using both a local cluster testbed and a large-scale simulation. Experimental results show that INFless outperforms state-of-theart systems by 2×-5× on system throughput, meeting the latency goals of ML services.
CCS CONCEPTS• Computer systems organization → Cloud computing.
A binary second order rational polynomial is adopted to simulate and extend the phase distribution on reference plane. In order to get the coefficients of the polynomial accurately, iterative least-square method based on the first order Taylor series expansion is used. The effect of the real reference plane profile error on measuring result is reduced by using extended unwrapped phase to substitute the original unwrapped phase. The effects of the random phase error and the system geometrical parameter error are decreased. The measuring accuracy of the system is improved. The principle of the 3D profile measuring system based on grating projection, the theoretic analysis for accurate estimate of phase distribution on reference plane, and the experimental results are presented.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.