Learning of Gaussian Processes in Distributed and Communication Limited Systems

Tavassolipour, Mostafa; Motahari, Seyed Abolfazl; Shalmani, Mohammad Taghi Manzuri

doi:10.48550/arxiv.1705.02627

Cited by 2 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some basic problems in machine learning such as classification, regression, hypothesis testing, etc. in distributed fashion are studied in [3], [33], [34]. Raginsky in [33] studied the classification and regression problem in distributed settings.…”

Section: Distributed Statistical Inferencementioning

confidence: 99%

“…Many learning algorithms can be modified to run distributively at several machines to perform a learning task. There are many papers that propose distributed (parallel) version of various learning algorithms [1], [2], [3], [4]. However, some learning algorithms could not be efficiently parallelized on distributed data.…”

Section: Introductionmentioning

confidence: 99%

“…In general, three models can be considered for data in distributed settings: horizontal model, where the data are distributed across samples; vertical model, where data are distributed over dimensions; and hybrid model which is the combination of both horizontal and vertical models. Designing distributed learning systems for the horizontal model has been addressed in many papers [3], [5]. In [6] and [7], the vertical model is studied from information theoretic point of view.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Learning of Tree-Structured Gaussian Graphical Models on Distributed Data Under Communication Constraints

Tavassolipour

Motahari

Shalmani

2019

IEEE Trans. Signal Process.

Self Cite

View full text Add to dashboard Cite

In this paper, learning of tree-structured Gaussian graphical models from distributed data is addressed. In our model, samples are stored in a set of distributed machines where each machine has access to only a subset of features. A central machine is then responsible for learning the structure based on received messages from the other nodes. We present a set of communication efficient strategies, which are theoretically proved to convey sufficient information for reliable learning of the structure. In particular, our analyses show that even if each machine sends only the signs of its local data samples to the central node, the tree structure can still be recovered with high accuracy. Our simulation results on both synthetic and real-world datasets show that our strategies achieve a desired accuracy in inferring the underlying structure, while spending a small budget on communication. Index Terms-Structure learning, Chow-Liu algorithm, Gaussian Graphical Model.!

show abstract

Section: Distributed Statistical Inferencementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning of Tree-Structured Gaussian Graphical Models on Distributed Data Under Communication Constraints

Tavassolipour

Motahari

Shalmani

2019

IEEE Trans. Signal Process.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Particularly, due to the product of experts, the aggregation scheme derives a factorized marginal likelihood for efficient training; and then it combines the experts' posterior distributions according to a certain aggregation criterion. In comparison to sparse approximations, the aggregation models (i) operate directly on the full training data, (ii) require no additional inducing or variational parameters and (iii) distribute the computations on individual experts for straightforward parallelization (Tavassolipour et al, 2017), thus scaling them to arbitrarily large training data. In comparison to typical local GPs (Snelson & Ghahramani, 2007;Park et al, 2011), the aggregations smooth out the ugly discontinuity by the product of posterior distributions from GP experts.…”

Section: Introductionmentioning

confidence: 99%

Generalized Robust Bayesian Committee Machine for Large-scale Gaussian Process Regression

Liu¹,

Cai²,

Wang³

et al. 2018

Preprint

View full text Add to dashboard Cite

In order to scale standard Gaussian process (GP) regression to large-scale datasets, aggregation models employ factorized training process and then combine predictions from distributed experts. The state-of-the-art aggregation models, however, either provide inconsistent predictions or require time-consuming aggregation process. We first prove the inconsistency of typical aggregations using disjoint or random data partition, and then present a consistent yet efficient aggregation model for large-scale GP. The proposed model inherits the advantages of aggregations, e.g., closed-form inference and aggregation, parallelization and distributed computing. Furthermore, theoretical and empirical analyses reveal that the new aggregation model performs better due to the consistent predictions that converge to the true underlying function when the training size approaches infinity.

show abstract

Learning of Gaussian Processes in Distributed and Communication Limited Systems

Cited by 2 publications

References 0 publications

Learning of Tree-Structured Gaussian Graphical Models on Distributed Data Under Communication Constraints

Learning of Tree-Structured Gaussian Graphical Models on Distributed Data Under Communication Constraints

Generalized Robust Bayesian Committee Machine for Large-scale Gaussian Process Regression

Contact Info

Product

Resources

About