Bighead: A Framework-Agnostic, End-to-End Machine Learning Platform

Brumbaugh, Eli; Kale, Atul; Luque, Alfredo; Nooraei, Bahador; Park, John; Puttaswamy, Krishna P. N.; Schiller, Kyle; Shapiro, E. A.; Shi, Conglei; Siegel, Aaron N.; Simha, N. V. R.; Bhushan, Mani; Sbrocca, Marie; Yao, Shi-Jing; Yoon, Patrick; Zanoyan, Varant; Zeng, Xiaohan; Zhu, Qiang; Cheong, Andrew; Du, Michelle Gu-Qian; Feng, Jeff; Handel, Nick; Hoh, Andrew; Hone, Jack; Hunter, Brad

doi:10.1109/dsaa.2019.00070

Cited by 10 publications

(17 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Figure 1 outlines the overall SLR process adopted in the study. [12][13][14][15][16][17][18][19][20][21][22][23][24] represent primary studies.…”

Section: Systematic Literature Review (Slr)mentioning

confidence: 99%

“…At this stage, the training process is terminated and all associated computing resources are released [14]. Before bringing the models into production, ensure that necessary burn-in tests are carried out to avoid initial model failures when put into production [17]. Proper planning of deployment process [28] can ensure a smooth transition from prototyping to deployment phase.…”

Section: Frameworkmentioning

confidence: 99%

“…Models, dependencies, artifacts, etc. need to be tracked and versioned to ensure reproducibility [17,28]. For instance, GitHub hashtags can be used for code versioning [28].…”

Section: Frameworkmentioning

confidence: 99%

“…For instance, GitHub hashtags can be used for code versioning [28]. It is highly recommended to maintain a registry containing entries for status of model life cycle and artifacts [17,28]. B.…”

Section: Frameworkmentioning

confidence: 99%

See 3 more Smart Citations

Architecting AI Deployment: A Systematic Review of State-of-the-Art and State-of-Practice Literature

John

Olsson

Bosch

2021

Lecture Notes in Business Information Processing

View full text Add to dashboard Cite

Companies across domains are rapidly engaged in shifting computational power and intelligence from centralized cloud to fully decentralized edges to maximize value delivery, strengthen security and reduce latency. However, most companies have only recently started pursuing this opportunity and are therefore at the early stage of the cloud-toedge transition. To provide an overview of AI deployment in the context of edge/cloud/hybrid architectures, we conduct a systematic literature review and a grey literature review. To advance understanding of how to integrate, deploy, operationalize and evolve AI models, we derive a framework from existing literature to accelerate the end-to-end deployment process. The framework is organized into five phases: Design, Integration, Deployment, Operation and Evolution. We make an attempt to analyze the extracted results by comparing and contrasting them to derive insights. The contribution of the paper is threefold. First, we conduct a systematic literature review in which we review the contemporary scientific literature and provide a detailed overview of the state-of-the-art of AI deployment. Second, we review the grey literature and present the state-of-practice and experience of practitioners while deploying AI models. Third, we present a framework derived from existing literature for the end-to-end deployment process and attempt to compare and contrast SLR and GLR results.

show abstract

“…Figure 1 outlines the overall SLR process adopted in the study. [12][13][14][15][16][17][18][19][20][21][22][23][24] represent primary studies.…”

Section: Systematic Literature Review (Slr)mentioning

confidence: 99%

Section: Frameworkmentioning

confidence: 99%

“…Models, dependencies, artifacts, etc. need to be tracked and versioned to ensure reproducibility [17,28]. For instance, GitHub hashtags can be used for code versioning [28].…”

Section: Frameworkmentioning

confidence: 99%

Section: Frameworkmentioning

confidence: 99%

See 2 more Smart Citations

Architecting AI Deployment: A Systematic Review of State-of-the-Art and State-of-Practice Literature

John

Olsson

Bosch

2021

Lecture Notes in Business Information Processing

View full text Add to dashboard Cite

show abstract

“…The hybrid architecture referred to in the study consists of a combination of cloud and edge architectures to deploy ML/DL models. -Data scientists need new ways of knowledge sharing [165] -Data scientists prefer to develop models alone [165] Tracking -Track models, dependencies [165], experiments [166], versions [167](eg: GitHub hash tags [165]), etc.…”

Section: Resultsmentioning

confidence: 99%

Design Methods and Processes for ML/DL models

John¹

View full text Add to dashboard Cite

Context: With the advent of Machine Learning (ML) and especially Deep Learning (DL) technology, companies are increasingly using Artificial Intelligence (AI) in systems, along with electronics and software. Nevertheless, the end-to-end process of developing, deploying and evolving ML and DL models in companies brings some challenges related to the design and scaling of these models. For example, access to and availability of data is often challenging, and activities such as collecting, cleaning, preprocessing, and storing data, as well as training, deploying and monitoring the model(s) are complex. Regardless of the level of expertise and/or access to data scientists, companies in all embedded systems domain struggle to build high-performing models due to a lack of established and systematic design methods and processes. Objective: The overall objective is to establish systematic and structured design methods and processes for the end-to-end process of developing, deploying and successfully evolving ML/DL models. Method: To achieve the objective, we conducted our research in close collaboration with companies in the embedded systems domain using different empirical research methods such as case study, action research and literature review. Results and Conclusions: This research provides six main results: First, it identifies the activities that companies undertake in parallel to develop, deploy and evolve ML/DL models, and the challenges associated with them. Second, it presents a conceptual framework for the continuous delivery of ML/DL models to accelerate AI-driven business in companies. Third, it presents a framework based on current literature to accelerate the end-to-end deployment process and advance knowledge on how to integrate, deploy and operationalize ML/DL models. Fourth, it develops a generic framework with five architectural alternatives for deploying ML/DL models at the edge. These architectural alternatives range from a centralized architecture that prioritizes (re)training in the cloud to a decentralized architecture that prioritizes (re)training at the edge. Fifth, it identifies key factors to help companies decide which architecture to choose for deploying ML/DL models. Finally, it explores how MLOps, as a practice that brings together data scientist teams and operations, ensures the continuous delivery and evolution of models.

show abstract

Metadata Representations for Queryable Repositories of Machine Learning Models

Li,

Kant,

Hai

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Machine learning (ML) practitioners and organizations are building model repositories of pre-trained models, referred to as model zoos. These model zoos contain metadata describing the properties of the ML models and datasets. The metadata serves crucial roles for reporting, auditing, ensuring reproducibility, and enhancing interpretability. Despite the growing adoption of descriptive formats like datasheets and model cards, the metadata available in existing model zoos remains notably limited. Moreover, existing formats have limited expressiveness, thus constraining the potential use of model repositories, extending their purpose beyond mere storage for pre-trained models. This paper proposes a unified metadata representation format for model zoos. We illustrate that comprehensive metadata enables a diverse range of applications, encompassing model search, reuse, comparison, and composition of ML models. We also detail the design and highlight the implementation of an advanced model zoo system built on top of our proposed metadata representation.

show abstract

Bighead: A Framework-Agnostic, End-to-End Machine Learning Platform

Cited by 10 publications

References 7 publications

Architecting AI Deployment: A Systematic Review of State-of-the-Art and State-of-Practice Literature

Architecting AI Deployment: A Systematic Review of State-of-the-Art and State-of-Practice Literature

Design Methods and Processes for ML/DL models

Metadata Representations for Queryable Repositories of Machine Learning Models

Contact Info

Product

Resources

About