Parallelized Variational EM for Latent Dirichlet Allocation: An Experimental Evaluation of Speed and Scalability

Mariote, Leonardo E.; Medeiros, Cláudia Bauzer; Torres, Ricardo da S.

doi:10.1109/icdmw.2007.33

Cited by 40 publications

(28 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yuan et al (2015) also presents an efficient constant time sampling algorithm for building big topic models. Variational inference can easily be parallelized (Nallapati et al, 2007;Zhai et al, 2012), but has high latency, which has been addressed by performing online updates (Hoffman et al, 2010) and taking stochastic gradients estimated by MCMC inference (Mimno et al, 2012). In this paper, we only focus on single-processor learning, but existing parallelization techniques (Newman et al, 2009) are applicable to our model.…”

Section: Related Workmentioning

confidence: 99%

Efficient Methods for Incorporating Knowledge into Topic Models

Yang¹,

Downey²,

Boyd-Graber³

2015

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Latent Dirichlet allocation (LDA) is a popular topic modeling technique for exploring hidden topics in text corpora. Increasingly, topic modeling needs to scale to larger topic spaces and use richer forms of prior knowledge, such as word correlations or document labels. However, inference is cumbersome for LDA models with prior knowledge. As a result, LDA models that use prior knowledge only work in small-scale scenarios. In this work, we propose a factor graph framework, Sparse Constrained LDA (SC-LDA), for efficiently incorporating prior knowledge into LDA. We evaluate SC-LDA's ability to incorporate word correlation knowledge and document label knowledge on three benchmark datasets. Compared to several baseline methods, SC-LDA achieves comparable performance but is significantly faster.

show abstract

Section: Related Workmentioning

confidence: 99%

Efficient Methods for Incorporating Knowledge into Topic Models

Yang¹,

Downey²,

Boyd-Graber³

2015

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

show abstract

“…[47] proposes parallel version of algorithms based on variational EM algorithm for LDA. Two settings of implementations are considered, one is in a multiprocessor architecture and the other is in a distributed environment.…”

Section: Parallel Learning Algorithmsmentioning

confidence: 99%

Probabilistic Models for Text Mining

Sun

Deng

2012

Mining Text Data

View full text Add to dashboard Cite

A number of probabilistic methods such as LDA, hidden Markov models, Markov random fields have arisen in recent years for probabilistic analysis of text data. This chapter provides an overview of a variety of probabilistic models for text mining. The chapter focuses more on the fundamental probabilistic techniques, and also covers their various applications to different text mining problems. Some examples of such applications include topic modeling, language modeling, document classification, document clustering, and information extraction.

show abstract

“…Among these three inference methods, VB inference and collapsed Gibbs sampling have already been parallelized by using PC clusters [8][11] [10]. However, both methods divide a given dataset into smaller subsets and process the subsets in parallel.…”

Section: Collapsed Variational Bayesian Inferencementioning

confidence: 99%

Accelerating Collapsed Variational Bayesian Inference for Latent Dirichlet Allocation with Nvidia CUDA Compatible Devices

Masada

Hamada

Shibata

et al. 2009

Next-Generation Applied Intelligence

View full text Add to dashboard Cite

Abstract. In this paper, we propose an acceleration of collapsed variational Bayesian (CVB) inference for latent Dirichlet allocation (LDA) by using Nvidia CUDA compatible devices. While LDA is an efficient Bayesian multi-topic document model, it requires complicated computations for parameter estimation in comparison with other simpler document models, e.g. probabilistic latent semantic indexing, etc. Therefore, we accelerate CVB inference, an efficient deterministic inference method for LDA, with Nvidia CUDA. In the evaluation experiments, we used a set of 50,000 documents and a set of 10,000 images. We could obtain inference results comparable to sequential CVB inference.

show abstract

Parallelized Variational EM for Latent Dirichlet Allocation: An Experimental Evaluation of Speed and Scalability

Abstract: Statistical topic models such as the Latent Dirichlet Allocation (LDA)

Cited by 40 publications

References 1 publication

Efficient Methods for Incorporating Knowledge into Topic Models

Efficient Methods for Incorporating Knowledge into Topic Models

Probabilistic Models for Text Mining

Accelerating Collapsed Variational Bayesian Inference for Latent Dirichlet Allocation with Nvidia CUDA Compatible Devices

Contact Info

Product

Resources

About