Managing ML pipelines

Abstract

The industrial machine learning pipeline requires iterating on model features, training and deploying models, and monitoring deployed models at scale. Feature stores were developed to manage and standardize the engineer's workflow in this end-to-end pipeline, focusing on traditional tabular feature data. In recent years, however, model development has shifted towards using self-supervised pretrained embeddings as model features. Managing these embeddings and the downstream systems that use them introduces new challenges with respect to managing embedding training data, measuring embedding quality, and monitoring downstream models that use embeddings. These challenges are largely unaddressed in standard feature stores. Our goal in this tutorial is to introduce the feature store system and discuss the challenges and current solutions to managing these new embedding-centric pipelines.

Tabi: An Efficient Multi-Level Inference System for Large Language Models

“…Selecting from available models is a standard practice in model-less inference [57], and not all models will be deployed. Besides, inference systems are not the only user of stored models [33,52]. We curate various models to show Tabi's generality, while the user-maintained repository size depends on users' deployment scale.…”

EnergAt: Fine-Grained Energy Attribution for Multi-Tenancy

“…High parallelism is prevalent in modern applications. Apart from traditional high-performance computing workloads, large numbers of parallel tasks in ML pipelines (e.g., feature stores [20,42,53]) can amount to 1/3 𝑟𝑑 of the total energy consumption, exceeding the amount of energy used by model training of large-scale jobs [54]. To precisely attribute CPU energy for a parallel application, one needs to obtain the CPU time for each of its tasks (processes and kernel threads) per socket.…”

Case Study - Feature Engineering Inspired by Domain Experts on Real World Medical Data

“…Moreover, information about extractor dependencies, intrinsic properties, and impact on prediction ability have a high value for the knowledge discovery pipeline. To utilise this information, a suitable framework is needed, also referred to as a feature store [14]. The concept of a feature store is now used by many artificial intelligence (AI) businesses to support their machine learning (ML) process [15].…”

“…The described KDFE process is time consuming, though we proved that it adds quantitative value to the researcher. If results and gained experience were collected from this study a knowledge foundation based on feature store [14] could be created that would support the development of an automated KDFE process. Qualitative aspects will be central in the design of the knowledge database that automated KDFE uses.…”