MLModelCI: An Automatic Cloud Platform for Efficient MLaaS

Abstract

MLModelCI provides multimedia researchers and developers with a one-stop platform for efficient machine learning (ML) services. The system leverages DevOps techniques to optimize, test, and manage models. It also containerizes and deploys these optimized and validated models as cloud services (MLaaS). In its essence, MLMod-elCI serves as a housekeeper to help users publish models. The models are first automatically converted to optimized formats for production purpose and then profiled under different settings (e.g., batch size and hardware). The profiling information can be used as guidelines for balancing the trade-off between performance and cost of MLaaS. Finally, the system dockerizes the models for ease of deployment to cloud environments. A key feature of MLModelCI is the implementation of a controller, which allows elastic evaluation which only utilizes idle workers while maintaining online service quality. Our system bridges the gap between current ML training and serving systems and thus free developers from manual and tedious work often associated with service deployment. We release the platform as an open-source project on GitHub under Apache 2.0 license, with the aim that it will facilitate and streamline more large-scale ML applications and research projects.

Active-Learning-as-a-Service: An Automatic and Efficient MLOps System for Data-Centric AI

“…From a model-centric view, we have model store systems [46], model continuous integration [53,35] tools, training platforms [22], deployment platforms [7], etc. Different from these systems, ALaaS is designed specifically for running AL tasks more efficiently.…”

“…Then the system will run AL tasks in an efficient pipeline manner. Meanwhile, more acceleration techniques such as data cache and batching [9,53,52] will be utilized to further speed up the AL process. In addition to that, our system also considers the accessibility and modularity so that non-experts can use AL strategies stored in our AL zoo with ease, and experts can propose more advanced AL strategies for more scenarios.…”

ModelPS: An Interactive and Collaborative Platform for Editing Pre-trained Models at Scale

“…(3) As a lightweight open-source system, ModelPS is highly customizable. It can work efficiently with popular frameworks, such as training system PyTorch Lighting [4], model deployment platform Clipper [3] and MLModelCI [23]. Those other platforms capable of pre-trained model editing, such as [2,17], are closed source and therefore can not be quickly extended to upgrade fast-evolving DNNs.…”

ModelCI-e: Enabling Continual Learning in Deep Learning Serving Systems

“…In this paper, we describe ModelCI-e (continuous integration [22] and evolution), an automated and efficient plugin that can be easily integrated into existing model serving systems (e.g., TensorFlow Serving, Clipper). We also conduct some preliminary studies to illustrate the potential challenges (e.g., interference of training and inference jobs in a cluster) for the system as well as the future optimization directions.…”