A GPU-accelerated Approximate Algorithm for Incremental Learning of Gaussian Mixture Model

Chen, Chunlei; Mu, Dejun; Zhang, Huixiang; Hong, Bin Na

doi:10.1109/ipdpsw.2012.236

Cited by 5 publications

(5 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The widely-used criterion is the variation of Q between steps, defined as: (6) letting i be an iteration counter, and ε a preset convergence criterion. Gaussian classifiers are one of the Hidden Markov Models applied to speech recognition.…”

Section: (3)mentioning

confidence: 99%

See 1 more Smart Citation

A Systematic Method for Detecting Parallelized Software Bottlenecks and Suggesting Modifications: The Case of the Expectation Maximization Algorithm

Oliveira¹,

Chella²,

Macedo³

et al. 2017

JSW

View full text Add to dashboard Cite

Parallelized algorithms can distribute the workload on the available multi-core processors. Graphical Processing Units (GPU) began to be used in general purpose computing thanks to its ability to simultaneously perform thousands of operations in their parallel coprocessors. Unfortunately, providing parallelized versions of typical sequential routines is not a trivial task. Even with the advent of CUDA, the NVIDIA's more intuitive solution for GPU programming, developers need to acquire a deep knowledge of GPU architecture and the rationale of the target algorithms to optimize resources usage and reduce processing time. This paper proposes a systematic method for analyzing parallelized algorithms and propose guidelines for CUDA code refactoring in such a way faster and more efficient software, regarding hardware resources consumption, could be constructed. One of such kind of software is Automatic Speech Recognition (ASR) systems. Mainstream approaches for ASR use the Expectation Maximization (EM) algorithm to train Gaussian Mixture Models (GMM) to provide an Acoustic Model for ASR. These training phase is usually extensive time-consuming and so it's well suited for a parallelized solution approach. We show the feasibility of our method identifying important issues in a literature's parallelized implementation of EM and further refactoring suggestion to enhance memory occupancy and decrease processing time. The results show a processing speedup of the EM algorithm around 40x (minimum) and 61x (maximum) when compared to the control version. The method was also effective in the improvement of the values for all the concerned performance metrics for GPU-based solutions.

show abstract

Section: (3)mentioning

confidence: 99%

“…For a large dataset, training processing time can be huge, especially in cases where there are many components. Despite such limitation, calculations for each of the data are independent and, so, fully parallelized [6].…”

Section: Phase 1 (Configuration and Execution Of Gpu-based Algorithm)mentioning

confidence: 99%

A Systematic Method for Detecting Parallelized Software Bottlenecks and Suggesting Modifications: The Case of the Expectation Maximization Algorithm

Oliveira¹,

Chella²,

Macedo³

et al. 2017

JSW

View full text Add to dashboard Cite

show abstract

“…The work focus on problems other than GMM. Chen et al (2012), the authors derive an algorithmic method for incremental GMM learning from a hypothesis-test and merging based algorithm. EM is not used.…”

Section: Em For Gmm Estimationmentioning

confidence: 99%

Parallel Implementation of Expectation-Maximisation Algorithm for the Training of Gaussian Mixture Models

Araújo¹,

Macedo²,

Chella³

et al. 2014

Journal of Computer Science

View full text Add to dashboard Cite

Most machine learning algorithms need to handle large data sets. This feature often leads to limitations on processing time and memory. The Expectation-Maximization (EM) is one of such algorithms, which is used to train one of the most commonly used parametric statistical models, the Gaussian Mixture Models (GMM). All steps of the algorithm are potentially parallelizable once they iterate over the entire data set. In this study, we propose a parallel implementation of EM for training GMM using CUDA. Experiments are performed with a UCI dataset and results show a speedup of 7 if compared to the sequential version. We have also carried out modifications to the code in order to provide better access to global memory and shared memory usage. We have achieved up to 56.4% of achieved occupancy, regardless the number of Gaussians considered in the set of experiments.

show abstract

“…Our previous work revealed that the existing incremental clustering algorithms are confronted with an accuracy-parallelism dilemma [ 5 , 6 ]. In this predicament, the governing factor is the evolving granularity of the incremental clustering algorithm.…”

Section: Introductionmentioning

confidence: 99%

“…The algorithm of [ 21 ] maintains the inherent parallelism. However, its clustering accuracy exhibits degradation by order of magnitude, compared to its batch-mode counterpart (the standard EM algorithm of GMM) [ 5 ]. Moreover, we qualitatively analyzed the reason why block-wise pattern tends to induce accuracy degradation in our previous work [ 6 ].…”

Section: Introductionmentioning

confidence: 99%

On the Accuracy and Parallelism of GPGPU-Powered Incremental Clustering Algorithms

Chen

Zhang

et al. 2017

Computational Intelligence and Neuroscience

Self Cite

View full text Add to dashboard Cite

Incremental clustering algorithms play a vital role in various applications such as massive data analysis and real-time data processing. Typical application scenarios of incremental clustering raise high demand on computing power of the hardware platform. Parallel computing is a common solution to meet this demand. Moreover, General Purpose Graphic Processing Unit (GPGPU) is a promising parallel computing device. Nevertheless, the incremental clustering algorithm is facing a dilemma between clustering accuracy and parallelism when they are powered by GPGPU. We formally analyzed the cause of this dilemma. First, we formalized concepts relevant to incremental clustering like evolving granularity. Second, we formally proved two theorems. The first theorem proves the relation between clustering accuracy and evolving granularity. Additionally, this theorem analyzes the upper and lower bounds of different-to-same mis-affiliation. Fewer occurrences of such mis-affiliation mean higher accuracy. The second theorem reveals the relation between parallelism and evolving granularity. Smaller work-depth means superior parallelism. Through the proofs, we conclude that accuracy of an incremental clustering algorithm is negatively related to evolving granularity while parallelism is positively related to the granularity. Thus the contradictory relations cause the dilemma. Finally, we validated the relations through a demo algorithm. Experiment results verified theoretical conclusions.

show abstract

A GPU-accelerated Approximate Algorithm for Incremental Learning of Gaussian Mixture Model

Cited by 5 publications

References 14 publications

A Systematic Method for Detecting Parallelized Software Bottlenecks and Suggesting Modifications: The Case of the Expectation Maximization Algorithm

A Systematic Method for Detecting Parallelized Software Bottlenecks and Suggesting Modifications: The Case of the Expectation Maximization Algorithm

Parallel Implementation of Expectation-Maximisation Algorithm for the Training of Gaussian Mixture Models

On the Accuracy and Parallelism of GPGPU-Powered Incremental Clustering Algorithms

Contact Info

Product

Resources

About