The BUDS Language for Distributed Bayesian Machine Learning

Gao, Zekai J.; Luo, Shangyu; Perez, Luis L.; Jermaine, Chris

doi:10.1145/3035918.3035937

Cited by 24 publications

(8 citation statements)

References 27 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly, Sys-temML [9] expresses machine learning algorithms by a simplified R and python like-language, and automatically translates the program into execution plan on top of Spark. BUDS [26] is a language for Bayesian machine learning, specifically, Markov chain simulation, allowing distributed computation on types of sets, maps, vectors and matrices. In addition, Datalog and its extensions [10,22,36] are also used to integrate statistical and machine learning into data management systems.…”

Section: Related Workmentioning

confidence: 99%

Towards Expectation-Maximization by SQL in RDBMS

Zhao

Rong

et al. 2021

Preprint

View full text Add to dashboard Cite

Integrating machine learning techniques into RDBMSs is an important task since there are many real applications that require modeling (e.g., business intelligence, strategic analysis) as well as querying data in RDBMSs. Without integration, it needs to export the data from RDBMSs to build a model using specialized machine learning toolkits and frameworks, and import the model trained back to RDBMSs for further querying. Such a process is not desirable since it is time-consuming and needs to repeat when data is changed. To support machine learning in RDBMSs, there are proposals that are platform-specific with limited functionalities to support certain modeling. In this paper, we provide an SQL solution that has the potential to support different machine learning modelings. As an example, we study how to support unsupervised probabilistic modeling, that has a wide range of applications in clustering, density estimation and data summarization, and focus on Expectation-Maximization (EM) algorithms, which is a general technique for finding maximum likelihood estimators. To train a model by EM, it needs to update the model parameters by an E-step and an M-step in a while-loop iteratively until it converges to a level controled by some threshold or repeats a certain number of iterations. To support EM in RDBMSs, we show our answers to the matrix/vectors representations in RDBMSs, the relational algebra operations to support the linear algebra operations required by EM, parameters update by relational algebra, and the support of a while-loop. It is important to note that the SQL'99 recursion cannot be used to handle such a while-loop since the M-step is non-monotonic. In addition, assume that a model has been trained by an EM algorithm, we further design an automatic in-database model maintenance mechanism to maintain the model when the underlying training data changes. We have conducted experimental studies and will report our findings in this paper.

show abstract

Section: Related Workmentioning

confidence: 99%

Towards Expectation-Maximization by SQL in RDBMS

Zhao

Rong

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Another related research thread includes the implementation of linear algebra systems on data management systems [1], [24]. There is increasing interest in building systems with the aim to achieve closer integration of ML with data management [1], [6], [9], [14], [20], [19], [13]. In the context of nonlinear models, [28] is concerned with Bayesian Markov Chain Monte Carlo as applied to Factorization Machine (FM) models and [33] present the algorithms for Support Vector Machine (SVM).…”

Section: Related Workmentioning

confidence: 99%

Nonlinear Models Over Normalized Data

Cheng

Koudas

2019

2019 IEEE 35th International Conference on Data Engineering (ICDE)

View full text Add to dashboard Cite

Machine Learning (ML) applications are proliferating in the enterprise. Relational data which are prevalent in enterprise applications are typically normalized; as a result, data has to be denormalized via primary/foreign-key joins to be provided as input to ML algorithms. In this paper, we study the implementation of popular nonlinear ML models, Gaussian Mixture Models (GMM) and Neural Networks (NN), over normalized data addressing both cases of binary and multiway joins over normalized relations. For the case of GMM, we show how it is possible to decompose computation in a systematic way both for binary joins and for multi-way joins to construct mixture models. We demonstrate that by factoring the computation, one can conduct the training of the models much faster compared to other applicable approaches, without any loss in accuracy. For the case of NN, we propose algorithms to train the network taking normalized data as the input. Similarly, we present algorithms that can conduct the training of the network in a factorized way and offer performance advantages. The redundancy introduced by denormalization can be exploited for certain types of activation functions. However, we demonstrate that attempting to explore this redundancy is helpful up to a certain point; exploring redundancy at higher layers of the network will always result in increased costs and is not recommended. We present the results of a thorough experimental evaluation, varying several parameters of the input relations involved and demonstrate that our proposals for the training of GMM and NN yield drastic performance improvements typically starting at 100%, which become increasingly higher as parameters of the underlying data vary, without any loss in accuracy.

show abstract

“…Machine learning frameworks SystemML [15], Ten-sorFlow [12], PyTorch [32], Mahout Samsara [1] and BUDS [22] provide domain specific languages (DSLs) and APIs that support linear algebra operations and data structures, probability distribution and/or deep learning functions, as well as useful ML-centric features, such as automatic differentation. These systems are more than ML libraries as they apply algebraic rewrites and operator fusion [20], [16] to optimize users' code.…”

Section: Related Workmentioning

confidence: 99%

sql4ml A declarative end-to-end workflow for machine learning

Makrynioti,

Ley-Wild,

Vassalos

2019

Preprint

View full text Add to dashboard Cite

We present sql4ml , a system for expressing supervised machine learning (ML) models in SQL and automatically training them in TensorFlow. The primary motivation for this work stems from the observation that in many data science tasks there is a back-and-forth between a relational database that stores the data and a machine learning framework. Data preprocessing and feature engineering typically happen in a database, whereas learning is usually executed in separate ML libraries. This fragmented workflow requires from the users to juggle between different programming paradigms and software systems. With sql4ml the user can express both feature engineering and ML algorithms in SQL, while the system translates this code to an appropriate representation for training inside a machine learning framework. We describe our translation method, present experimental results from applying it on three well-known ML algorithms and discuss the usability benefits from concentrating the entire workflow on the database side.

show abstract

The BUDS Language for Distributed Bayesian Machine Learning

Cited by 24 publications

References 27 publications

Towards Expectation-Maximization by SQL in RDBMS

Towards Expectation-Maximization by SQL in RDBMS

Nonlinear Models Over Normalized Data

sql4ml A declarative end-to-end workflow for machine learning

Contact Info

Product

Resources

About