Towards MLOps: A Case Study of ML Pipeline Platform

Zhou, Yue; Yu, Yue; Ding, Bo

doi:10.1109/icaice51518.2020.00102

Cited by 50 publications

(33 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…14,[16][17][18][19][20][21][22][23][24] Our conceptual prototypes extend this work by proposing an integrated MLOps framework for the entire predictive model development lifecycle, including a core panel of features that would otherwise require configuring multiple services into a custom pipeline. 29,67 Unlike efforts by Google, 32,33 Facebook, 31 IBM, 30,46 and others, our work complements existing opensource MLOps frameworks, such as MLFlow, 35 by providing a potential GUI for its primarily command-line driven functionality.…”

Section: Discussionmentioning

confidence: 99%

A Graphical Toolkit for Longitudinal Dataset Maintenance and Predictive Model Training in Health Care

et al. 2022

View full text Add to dashboard Cite

Background Predictive analytic models, including machine learning (ML) models, are increasingly integrated into electronic health record (EHR)-based decision support tools for clinicians. These models have the potential to improve care, but are challenging to internally validate, implement, and maintain over the long term. Principles of ML operations (MLOps) may inform development of infrastructure to support the entire ML lifecycle, from feature selection to long-term model deployment and retraining. Objectives This study aimed to present the conceptual prototypes for a novel predictive model management system and to evaluate the acceptability of the system among three groups of end users. Methods Based on principles of user-centered software design, human-computer interaction, and ethical design, we created graphical prototypes of a web-based MLOps interface to support the construction, deployment, and maintenance of models using EHR data. To assess the acceptability of the interface, we conducted semistructured user interviews with three groups of users (health informaticians, clinical and data stakeholders, chief information officers) and evaluated preliminary usability using the System Usability Scale (SUS). We subsequently revised prototypes based on user input and developed user case studies. Results Our prototypes include design frameworks for feature selection, model training, deployment, long-term maintenance, visualization over time, and cross-functional collaboration. Users were able to complete 71% of prompted tasks without assistance. The average SUS score of the initial prototype was 75.8 out of 100, translating to a percentile range of 70 to 79, a letter grade of B, and an adjective rating of “good.” We reviewed persona-based case studies that illustrate functionalities of this novel prototype. Conclusion The initial graphical prototypes of this MLOps system are preliminarily usable and demonstrate an unmet need within the clinical informatics landscape.

show abstract

Section: Discussionmentioning

confidence: 99%

A Graphical Toolkit for Longitudinal Dataset Maintenance and Predictive Model Training in Health Care

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Our selected ML model is networkspecific. Today, the domain of taking an ML method from the laboratory to an actual environment and ensuring that it can operate smoothly in the real world is called MLOps [40]. It is well-known that ML models heavily depend on the quality of data and are tailored to the targeted applications (in our case a specific network configuration).…”

Section: Discussionmentioning

confidence: 99%

Machine Learning-Based Network Status Detection and Fault Localization

Mohammed

Côté

et al. 2021

IEEE Trans. Instrum. Meas.

View full text Add to dashboard Cite

Although the autonomous detection of network status and localization of network faults can be a valuable tool for network and service operators, very few works have investigated this subject. As a result in today's networks, fault detection and localization remains a mostly-manual process. In this paper, we propose a Machine Learning (ML) method that can automatically detect the status of a network and localize faults. Our method uses Decision Tree, Gradient Boosting (GB), and XGBoost (XGB) ML algorithms to detect the network status as Normal, Congestion, and Network Fault. In comparison, existing related work can at best classify the network status as faulty or non-faulty. Experimental results show that our method yields accuracies of up to 99% on a dataset collected through an emulated network.

show abstract

“…This perspective shifts the focus from just ML algorithms to also include other important aspects of ML model development and operations in production, such as data management and serving infrastructures [1]. Evidence of the integration between SE approaches and ML workflow is in MLOps (machine learning operations), a term used to show the extension of DevOps philosophy of increased agility and automation to the ML workflows [8]. In support of the latter, different tools are used to provide automation in ML workflows.…”

Section: Background and Related Workmentioning

confidence: 99%

Practices and Infrastructures for ML Systems – An Interview Study in Finnish Organizations

Muiruri¹,

Lwakatare²,

Nurminen³

et al. 2022

Preprint

View full text Add to dashboard Cite

<div> <div> <div> <p>The best practices and infrastructures for developing and maintaining machine learning (ML) enabled software systems are often reported by large and experienced data-driven organizations. However, little is known about the state of practice across other organizations. Using interviews, we investigated practices and tool-chains for ML-enabled systems from 16 organizations in various domains. Our study makes three broad observations related to data management practices, monitoring practices and automation practices in ML model training, and serving workflows. These have limited number of generic practices and tools applicable across organizations in different domains. </p> </div> </div> </div>

show abstract

Towards MLOps: A Case Study of ML Pipeline Platform

Cited by 50 publications

References 17 publications

A Graphical Toolkit for Longitudinal Dataset Maintenance and Predictive Model Training in Health Care

A Graphical Toolkit for Longitudinal Dataset Maintenance and Predictive Model Training in Health Care

Machine Learning-Based Network Status Detection and Fault Localization

Practices and Infrastructures for ML Systems – An Interview Study in Finnish Organizations

Contact Info

Product

Resources

About