AMIDST: A Java toolbox for scalable probabilistic machine learning

Masegosa, Andrés R.; Martínez, Ana María; Ramos-López, Darío; Cabañas, Rafael; Salmerón, Antonio; Langseth, Helge; Nielsen, Thomas D.; Madsen, Anders L.

doi:10.1016/j.knosys.2018.09.019

Cited by 11 publications

(13 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lastly, we discuss the proposed modeling framework in a more general setting, linking model validation and concept drift analysis. The proposed methods are released as part of an open-source toolbox for scalable probabilistic machine learning (http://www.amidsttoolbox.com) [6,7,8].…”

Section: Introductionmentioning

confidence: 99%

Analyzing concept drift: A case study in the financial sector

Masegosa

Martínez

Ramos-López

et al. 2020

IDA

Self Cite

View full text Add to dashboard Cite

In this paper we present a method for exploratory data analysis of streaming data based on probabilistic graphical models (latent variable models). This method is illustrated by concept drift tracking, using financial client data from a European regional bank. For this particular setting, the analyzed data spans the period from April 2007 to March 2014, and therefore starts before the beginning of the financial crisis of 2008. The implied changes in the economic climate during this period manifests itself as concept drift in the underlying data generating distribution. We explore and analyze this financial client data using a probabilistic graphical modeling framework that provides an explicit representation of concept drift as an integral part of the model. We show how learning these types of models from data provides additional insight into the hidden mechanisms governing the drift in the domain. We present an iterative approach for identifying disparate factors that jointly account for the drift in the domain. This includes a semantic characterization of one of the main influencing drift factors. Based on the experiences and results obtained from analyzing the financial data, we discuss the applicability of the framework within a more general context.

show abstract

Section: Introductionmentioning

confidence: 99%

Analyzing concept drift: A case study in the financial sector

Masegosa

Martínez

Ramos-López

et al. 2020

IDA

Self Cite

View full text Add to dashboard Cite

show abstract

“…The R package ramidst (Salmeron et al 2016) links to the AMIDST (Masegosa et al 2019) toolbox for scalable, but only approximate inference algorithms for BNs written in Java (Gosling, Joy, Steele, and Bracha 2000). The package was recently removed from the CRAN repository for check issues, but the Java toolbox remains functional.…”

Section: Bayesnetbpmentioning

confidence: 99%

BayesNetBP: An R Package for Probabilistic Reasoning in Bayesian Networks

Yu¹,

Moharil²,

Blair³

2020

J. Stat. Soft.

View full text Add to dashboard Cite

The BayesNetBP package has been developed for probabilistic reasoning and visualization in Bayesian networks with nodes that are purely discrete, continuous or mixed (discrete and continuous). Probabilistic reasoning enables a user to absorb information into a Bayesian network and make queries about how the probabilities within the network change in light of new information. The package was developed in the R programming language and is freely available from the Comprehensive R Archive Network. A shiny app with Cytoscape widgets provides an interactive interface for evidence absorption, queries, and visualizations.

show abstract

“…Edward can exploit GPUs for parallelism. AMIDST [44] is a Java toolbox for scalable probabilistic machine learning and allows a user to build probabilistic graphical models and perform scalable inference. To process large data streams and large-scale datasets, AMIDST employs Apache Flink [23] and Apache Spark [58].…”

Section: Distributed Machine Learning Frameworkmentioning

confidence: 99%

“…Because gossiping can be stopped on the cluster nodes after a period of time and started again, DiSC can eciently recompute the family scores as new data are produced. Unlike AMIDST [44] that is designed for a streaming scenario where new data arrive continuously, we focus on stored datasets that may be updated over time but not in real-time. Suppose a new data block with some number of data instances is added to a cluster node.…”

Section: Recomputing Family Scores On New Datamentioning

confidence: 99%

Fast Approximate Score Computation on Large-Scale Distributed Data for Learning Multinomial Bayesian Networks

Katib

Rao

Barnard

et al. 2019

ACM Trans. Knowl. Discov. Data

View full text Add to dashboard Cite

In this paper, we focus on the problem of learning a Bayesian network over distributed data stored in a commodity cluster. Specically, we address the challenge of computing the scoring function over distributed data in an ecient and scalable manner, which is a fundamental task during learning. While exact score computation can be done using the MapReduce-style computation, our goal is to compute approximate scores much faster with probabilistic error bounds and in a scalable manner. We propose a novel approach which is designed to achieve: (a) decentralized score computation using the principle of gossiping; (b) lower resource consumption via a probabilistic approach for maintaining scores using the properties of a Markov chain; and (c) eective distribution of tasks during score computation (on large datasets) by synergistically combining well-known hashing techniques. We conduct theoretical analysis of our approach in terms of convergence speed of the statistics required for score computation, and memory and network bandwidth consumption. We also discuss how our approach is capable of eciently recomputing scores when new data are available. We conducted a comprehensive evaluation of our approach and compared with the MapReduce-style computation using datasets of dierent characteristics on a 16-node cluster. When the MapReduce-style computation provided exact statistics for score computation, it was nearly 10 times slower than our approach. Although it ran faster on randomly sampled datasets than on the entire datasets, it performed worse than our approach in terms of accuracy. Our approach achieved high accuracy (below 6% average relative error) in estimating the statistics for approximate score computation on all the tested datasets. In conclusion, it provides a feasible tradeo between computation time and accuracy for fast approximate score computation on large-scale distributed data.

show abstract

AMIDST: A Java toolbox for scalable probabilistic machine learning

Cited by 11 publications

References 9 publications

Analyzing concept drift: A case study in the financial sector

Analyzing concept drift: A case study in the financial sector

BayesNetBP: An R Package for Probabilistic Reasoning in Bayesian Networks

Fast Approximate Score Computation on Large-Scale Distributed Data for Learning Multinomial Bayesian Networks

Contact Info

Product

Resources

About