From DevOps to MLOps: Overview and Application to Electricity Market Forecasting

Subramanya, Rakshith; Sierla, Seppo; Vyatkin, Valeriy

doi:10.3390/app12199851

Cited by 26 publications

(8 citation statements)

References 156 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such monitoring can be achieved by implementing ML Operations (MLOps) to track model performances through versioning across training—retraining, and the associated impact from data drifts. The MLOps practices provide the management of ML models with faster model building‐deployment, high quality, reproducibility, and end‐to‐end tracking (Subramanya et al, 2022). Our developed prototype application has basic elements of MLOps with high prospects of automation.…”

Section: Discussionmentioning

confidence: 99%

“…Such monitoring can be achieved by implementing ML Operations (MLOps) to track model performances through versioning across training-retraining, and the associated impact from data drifts. The MLOps practices provide the management of ML models with faster model building-deployment, high quality, reproducibility, and end-to-end tracking(Subramanya et al, 2022). Our developed prototype application has basic elements of MLOps with high prospects of automation.The component of analytical variability prediction can be adapted and applied to enable other areas such as process analytical technologies, wherein chemometric modelling is applied and variability among the calibration sets is known to impact the accuracies.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Digital by design approach to develop a universal deep learning AI architecture for automatic chromatographic peak integration

Satwekar

Panda

Nandula

et al. 2023

Biotech & Bioengineering

View full text Add to dashboard Cite

Chromatographic data processing has garnered attention due to multiple Food and Drug Administration 483 citations and warning letters, highlighting the need for a robust technological solution. The healthcare industry has the potential to greatly benefit from the adoption of digital technologies, but the process of implementing these technologies can be slow and complex. This article presents a "Digital by Design" managerial approach, adapted from pharmaceutical quality by design principles, for designing and implementing an artificial intelligence (AI)-based solution for chromatography peak integration process in the healthcare industry.We report the use of a convolutional neural network model to predict analytical variability for integrating chromatography peaks and propose a potential GxP framework for using AI in the healthcare industry that includes elements on data management, model management, and human-in-the-loop processes. The component on analytical variability prediction has a great potential to enable Industry 4.0 objectives on real-time release testing, automated quality control, and continuous manufacturing.

show abstract

Section: Discussionmentioning

confidence: 99%

mentioning

confidence: 99%

Digital by design approach to develop a universal deep learning AI architecture for automatic chromatographic peak integration

Satwekar

Panda

Nandula

et al. 2023

Biotech & Bioengineering

View full text Add to dashboard Cite

show abstract

“…The evolution of AI in healthcare has led to various significant advancements, many of which are integrated into existing MLOps frameworks ( 1 ). A plethora of research exists, focusing on improving data quality, model training, evaluation, and deployment in the healthcare domain ( 2 , 3 ).…”

Section: Introductionmentioning

confidence: 99%

Resilience-aware MLOps for AI-based medical diagnostic system

Moskalenko,

Kharchenko

2024

Front. Public Health

View full text Add to dashboard Cite

BackgroundThe healthcare sector demands a higher degree of responsibility, trustworthiness, and accountability when implementing Artificial Intelligence (AI) systems. Machine learning operations (MLOps) for AI-based medical diagnostic systems are primarily focused on aspects such as data quality and confidentiality, bias reduction, model deployment, performance monitoring, and continuous improvement. However, so far, MLOps techniques do not take into account the need to provide resilience to disturbances such as adversarial attacks, including fault injections, and drift, including out-of-distribution. This article is concerned with the MLOps methodology that incorporates the steps necessary to increase the resilience of an AI-based medical diagnostic system against various kinds of disruptive influences.MethodsPost-hoc resilience optimization, post-hoc predictive uncertainty calibration, uncertainty monitoring, and graceful degradation are incorporated as additional stages in MLOps. To optimize the resilience of the AI based medical diagnostic system, additional components in the form of adapters and meta-adapters are utilized. These components are fine-tuned during meta-training based on the results of adaptation to synthetic disturbances. Furthermore, an additional model is introduced for post-hoc calibration of predictive uncertainty. This model is trained using both in-distribution and out-of-distribution data to refine predictive confidence during the inference mode.ResultsThe structure of resilience-aware MLOps for medical diagnostic systems has been proposed. Experimentally confirmed increase of robustness and speed of adaptation for medical image recognition system during several intervals of the system’s life cycle due to the use of resilience optimization and uncertainty calibration stages. The experiments were performed on the DermaMNIST dataset, BloodMNIST and PathMNIST. ResNet-18 as a representative of convolutional networks and MedViT-T as a representative of visual transformers are considered. It is worth noting that transformers exhibited lower resilience than convolutional networks, although this observation may be attributed to potential imperfections in the architecture of adapters and meta-adapters.СonclusionThe main novelty of the suggested resilience-aware MLOps methodology and structure lie in the separating possibilities and activities on creating a basic model for normal operating conditions and ensuring its resilience and trustworthiness. This is significant for the medical applications as the developer of the basic model should devote more time to comprehending medical field and the diagnostic task at hand, rather than specializing in system resilience. Resilience optimization increases robustness to disturbances and speed of adaptation. Calibrated confidences ensure the recognition of a portion of unabsorbed disturbances to mitigate their impact, thereby enhancing trustworthiness.

show abstract

“…MLOps constitute a set of tools and mechanisms able to enhance the collaboration between data scientists and IT professionals in applying and maintaining ML models [28,29]. In turn, MLOps practices are in the position to assist the risk management associated with deploying ML models in production by providing traceability, monitoring, and testing capabilities [28,30,31]. In general, two major issues are considered in designing MLOps procedures [28,30,32].…”

Section: Introductionmentioning

confidence: 99%

Structure Learning and Hyperparameter Optimization Using an Automated Machine Learning (AutoML) Pipeline

et al. 2023

View full text Add to dashboard Cite

In this paper, we built an automated machine learning (AutoML) pipeline for structure-based learning and hyperparameter optimization purposes. The pipeline consists of three main automated stages. The first carries out the collection and preprocessing of the dataset from the Kaggle database through the Kaggle API. The second utilizes the Keras-Bayesian optimization tuning library to perform hyperparameter optimization. The third focuses on the training process of the machine learning (ML) model using the hyperparameter values estimated in the previous stage, and its evaluation is performed on the testing data by implementing the Neptune AI. The main technologies used to develop a stable and reusable machine learning pipeline are the popular Git version control system, the Google cloud virtual machine, the Jenkins server, the Docker containerization technology, and the Ngrok reverse proxy tool. The latter can securely publish the local Jenkins address as public through the internet. As such, some parts of the proposed pipeline are taken from the thematic area of machine learning operations (MLOps), resulting in a hybrid software scheme. The machine learning model was used to evaluate the pipeline, which is a multilayer perceptron (MLP) that combines typical dense, as well as polynomial, layers. The simulation results show that the proposed pipeline exhibits a reliable and accurate performance while managing to boost the network’s performance in classification tasks.

show abstract

From DevOps to MLOps: Overview and Application to Electricity Market Forecasting

Cited by 26 publications

References 156 publications

Digital by design approach to develop a universal deep learning AI architecture for automatic chromatographic peak integration

Digital by design approach to develop a universal deep learning AI architecture for automatic chromatographic peak integration

Resilience-aware MLOps for AI-based medical diagnostic system

Structure Learning and Hyperparameter Optimization Using an Automated Machine Learning (AutoML) Pipeline

Contact Info

Product

Resources

About