Accelerating pattern matching in neuromorphic text recognition system using Intel Xeon Phi coprocessor

Ahmed, Khadeer; Qiu, Qinru; Malani, Parth; Tamhankar, Mangesh

doi:10.1109/ijcnn.2014.6889777

Cited by 6 publications

(3 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The performance gain on Intel Xeon Phi 7110p for a model called Brain-State-in-a-Box (BSB) used for text recognition is studied by Ahmed et al in [2]. The authors report about two-fold speedup for the co-processor compared to a CPU with 16 cores when parallelizing the algorithm.…”

Section: Machine Learning Targeting Intel Xeon Phimentioning

confidence: 99%

CHAOS: a parallelization scheme for training convolutional neural networks on Intel Xeon Phi

et al. 2017

View full text Add to dashboard Cite

Deep learning is an important component of Big Data analytic tools and intelligent applications, such as self-driving cars, computer vision, speech recognition, or precision medicine. However, the training process is computationally intensive and often requires a large amount of time if performed sequentially. Modern parallel computing systems provide the capability to reduce the required training time of deep neural networks. In this paper, we present our parallelization scheme for training convolutional neural networks (CNN) named Controlled Hogwild with Arbitrary Order of Synchronization (CHAOS). Major features of CHAOS include the support for thread and vector parallelism, non-instant updates of weight parameters during back-propagation without a significant delay, and implicit synchronization in arbitrary order. CHAOS is tailored for parallel computing systems that are accelerated with the Intel Xeon Phi. We evaluate our parallelization approach empirically using measurement techniques and performance modeling for various numbers of threads and CNN architectures. Experimental results for the MNIST dataset of handwritten digits using the total number of threads on the Xeon Phi show speedups of up to 103× compared to the execution on one thread of the Xeon Phi, 14× compared to the sequential execution on Intel Xeon E5, and 58× compared to the sequential execution on Intel Core i5.

show abstract

Section: Machine Learning Targeting Intel Xeon Phimentioning

confidence: 99%

CHAOS: a parallelization scheme for training convolutional neural networks on Intel Xeon Phi

et al. 2017

View full text Add to dashboard Cite

show abstract

“…A Brain-State-In-A-Box neural network was optimized and evaluated on an Intel Xeon Phi 7110P by Khadeer et al [15] achieving about two-fold speed up relative to a CPU with 16 cores.…”

Section: A Machine Learning and Intel Xeon Phimentioning

confidence: 99%

“…However, not much research related to the Intel Xeon Phi and Deep Learning has been done so far. A study by Jin et al [15] target unsupervised learning of Restricted Boltzmann Machines and Sparse Auto Encoders. Evaluation performed on an Intel Xeon Phi 5110P resulted with a speed up of 7 to 10 times compared to an Intel Xeon E5620.…”

Section: Introductionmentioning

confidence: 99%

The Potential of the Intel (R) Xeon Phi for Supervised Deep Learning

Viebke

Pllana

2015

2015 IEEE 17th International Conference on High Performance Computing and Communications, 2015 IEEE 7th International Symposium

View full text Add to dashboard Cite

Abstract-Supervised learning of Convolutional Neural Networks (CNNs), also known as supervised Deep Learning, is a computationally demanding process. To find the most suitable parameters of a network for a given application, numerous training sessions are required. Therefore, reducing the training time per session is essential to fully utilize CNNs in practice. While numerous research groups have addressed the training of CNNs using GPUs, so far not much attention has been paid to the Intel Xeon Phi coprocessor. In this paper we investigate empirically and theoretically the potential of the Intel Xeon Phi for supervised learning of CNNs. We design and implement a parallelization scheme named CHAOS that exploits both the thread-and SIMD-parallelism of the coprocessor. Our approach is evaluated on the Intel Xeon Phi 7120P using the MNIST dataset of handwritten digits for various thread counts and CNN architectures. Results show a 103.5x speed up when training our large network for 15 epochs using 244 threads, compared to one thread on the coprocessor. Moreover, we develop a performance model and use it to assess our implementation and answer what-if questions.

show abstract