Tensor Relational Algebra for Machine Learning System Design

Yuan, Binhang; Jankov, Dimitrije; Zou, Jia; Tang, Yuxin; Bourgeois, Daniel C.; Jermaine, Chris

doi:10.48550/arxiv.2009.00524

Cited by 4 publications

(11 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…; (3) Apply the relational algebra operators nested with these UDFs for performing linear algebra computations. Based on the above ideas, tensor relational algebra (TRA) [66] further introduces a set of tensor-oriented relational operations, such as tile, concat, rekey, transform, join, aggregation, selection, etc. We found that most ML workloads can be decomposed into linear algebra operations that are further represented in such TRA.…”

Section: Background and Related Work 21 ML Model Inferences As Queriesmentioning

confidence: 99%

“…For example, matrix multiplication is a join followed by aggregation [12,66]. The join pairs two blocks from the two tensors if the first block's column index equals the second's row index.…”

Section: Background and Related Work 21 ML Model Inferences As Queriesmentioning

confidence: 99%

“…Similarly, matrix addition is a join. In addition, matrix transpose is a rekey [66]; activations such as relu, tanh, and sigmoid are transforms; softmax and normalization can be represented as an aggregation followed by a transform.…”

Section: Background and Related Work 21 ML Model Inferences As Queriesmentioning

confidence: 99%

“…Therefore, it is imperative to investigate the serving of deep learning models natively from the relational database management system (RDBMS) [12,25,31,34,36,40,53,63,66]. RDBMS has a long history of optimizing the memory locality for computations (i.e., queries), whether the working set size exceeds memory capacity or not, through effective buffer pool management.…”

Section: Introductionmentioning

confidence: 99%

“…To address these challenges, in this work, we propose a novel RDBMS storage design optimized for tensors and DNN inference workloads. We mainly leverage our previous works on Tensor Relational Algebra [34,66] to map deep learning computations to relational algebra expressions. A tensor is partitioned and stored as a set of tensor blocks of equivalent shape, where each block contains the metadata that specifies its position in the tensor.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Serving Deep Learning Models with Deduplication from Relational Databases

Zhou¹,

Chen²,

Das³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Serving deep learning models from relational databases brings significant benefits. First, features extracted from databases do not need to be transferred to any decoupled deep learning systems for inferences, and thus the system management overhead can be significantly reduced. Second, in a relational database, data management along the storage hierarchy is fully integrated with query processing, and thus it can continue model serving even if the working set size exceeds the available memory. Applying model deduplication can greatly reduce the storage space, memory footprint, cache misses, and inference latency. However, existing data deduplication techniques are not applicable to the deep learning model serving applications in relational databases. They do not consider the impacts on model inference accuracy as well as the inconsistency between tensor blocks and database pages. This work proposed synergistic storage optimization techniques for duplication detection, page packing, and caching, to enhance database systems for model serving. Evaluation results show that our proposed techniques significantly improved the storage efficiency and the model inference latency, and serving models from relational databases outperformed existing deep learning frameworks when the working set size exceeds available memory.

show abstract

Section: Background and Related Work 21 ML Model Inferences As Queriesmentioning

confidence: 99%

Section: Background and Related Work 21 ML Model Inferences As Queriesmentioning

confidence: 99%

Section: Background and Related Work 21 ML Model Inferences As Queriesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Serving Deep Learning Models with Deduplication from Relational Databases

Zhou¹,

Chen²,

Das³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

End-to-end Optimization of Machine Learning Prediction Queries

Park

Saur

Banda

et al. 2022

Proceedings of the 2022 International Conference on Management of Data

View full text Add to dashboard Cite

Prediction queries are widely used across industries to perform advanced analytics and draw insights from data. They include a data processing part (e.g., for joining, filtering, cleaning, featurizing the datasets) and a machine learning (ML) part invoking one or more trained models to perform predictions. These parts have so far been optimized in isolation, leaving significant opportunities for optimization unexplored.We present Raven, a production-ready system for optimizing prediction queries. Raven follows the enterprise architectural trend of collocating data and ML runtimes. It relies on a unified intermediate representation that captures both data and ML operators in a single graph structure to unlock two families of optimizations. First, it employs logical optimizations that pass information between the data part (and the properties of the underlying data) and the ML part to optimize each other. Second, it introduces logical-tophysical transformations that allow operators to be executed on different runtimes (relational, ML, and DNN) and hardware (CPU, GPU). Novel data-driven optimizations determine the runtime to be used for each part of the query to achieve optimal performance. Our evaluation shows that Raven improves performance of prediction queries on Apache Spark and SQL Server by up to 13.1× and 330×, respectively. For complex models where GPU acceleration is beneficial, Raven provides up to 8× speedup compared to state-of-the-art systems.

show abstract

Benchmark of DNN Model Search at Deployment Time

Zhou,

Jain,

Wang

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Deep learning has become the most popular direction in machine learning and artificial intelligence. However, the preparation of training data, as well as model training, are often time-consuming and become the bottleneck of the end-to-end machine learning lifecycle. Reusing models for inferring a dataset can avoid the costs of retraining. However, when there are multiple candidate models, it is challenging to discover the right model for reuse. Although there exist a number of model sharing platforms such as ModelDB, TensorFlow Hub, PyTorch Hub, and DLHub, most of these systems require model uploaders to manually specify the details of each model and model downloaders to screen keyword search results for selecting a model. We are lacking a highly productive model search tool that selects models for deployment without the need for any manual inspection and/or labeled data from the target domain. This paper proposes multiple model search strategies including various similarity-based approaches and non-similarity-based approaches. We design, implement and evaluate these approaches on multiple model inference scenarios, including activity recognition, image recognition, text classification, natural language processing, and entity matching. The experimental evaluation showed that our proposed asymmetric similarity-based measurement, adaptivity, outperformed symmetric similarity-based measurements and nonsimilarity-based measurements in most of the workloads. CCS CONCEPTS• Information systems → Clustering and classification; Evaluation of retrieval results; • Computing methodologies → Neural networks.

show abstract

Tensor Relational Algebra for Machine Learning System Design

Cited by 4 publications

References 55 publications

Serving Deep Learning Models with Deduplication from Relational Databases

Serving Deep Learning Models with Deduplication from Relational Databases

End-to-end Optimization of Machine Learning Prediction Queries

Benchmark of DNN Model Search at Deployment Time

Contact Info

Product

Resources

About