Scaling Deep Learning workloads: NVIDIA DGX-1/Pascal and Intel Knights Landing

Gawande, Nitin; Daily, Jeff; Siegel, Charles; Tallent, Nathan R.; Vishnu, Abhinav

doi:10.1016/j.future.2018.04.073

Cited by 23 publications

(11 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…KNL and KNM Phis have higher memory capacity than GPU, which allows them to run even those codes which cannot run on GPU. 62,93 The strength of GPU lies in use of massive multithreading and high memory bandwidth. Also, texture units bring large speedup for graphics applications.…”

Section: Discussionmentioning

confidence: 99%

“…Overall, Phi does not provide comparable performance to CPU as a stand-alone shared memory processor. Thread-affinity strategy Balanced, 4,14,20,21,23,26,36 scatter, 37 compact, 92 no single winner 13,84,94 Memory mode Cache, 60,62 flat, 55,90,96 hybrid (none), no single winner [10][11][12]52,54,57,97 Interconnect clustering mode All-to-all (none), quadrant, 11,62 sub-NUMA, 10,55,57,96,97 no single winner 52,96…”

Section: Gaining Insights Into Phi Architecturementioning

confidence: 99%

“…Vectorization-related Difficulties in or imperfect vectorization, 2,22,25 incompatibility of SSE-style vectorization 2 Poor scalability with increasing threads/nodes 2, 16,20,29,37,57,62,66,84 Slow operations reduction operation, 86 gather-scatter operations, 82 divide operations 13,26 Data dependency 99 Load-imbalance between threads of Phi, 13,17,23,50 CPU and Phi 18,43,50,76 Contention on shared components ring stop, 29,37,84 disk-access 37 Others lack of support for vector atomic instructions, 37 thread-creation overhead, 30 TLB misses 42,90 intrinsic functions for introducing vector variables and processes. This approach provides better performance compared to the compiler performing auto-vectorization.…”

Section: Comparison With Cpumentioning

confidence: 99%

“…Gawande et al 62 In summary, GPU provides higher raw performance and NVLink is crucial for GPU scaling. Also, KNL can provide performance/watt comparable to that of GPU.…”

Section: Comparison With Gpumentioning

confidence: 99%

See 3 more Smart Citations

A survey on evaluating and optimizing performance of Intel Xeon Phi

Mittal

2020

Concurrency and Computation

View full text Add to dashboard Cite

Summary Intel's Xeon Phi combines the parallel processing power of a many‐core accelerator with the programming ease of CPUs. In this paper, we present a survey of works that study the architecture of Phi and use it as an accelerator for a broad range of applications. We review performance optimization strategies as well as the factors that bottleneck the performance of Phi. We also review works that perform comparison or collaborative execution of Phi with CPUs and GPUs. This paper will be useful for researchers and developers in the area of computer‐architecture and high‐performance computing.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Gaining Insights Into Phi Architecturementioning

confidence: 99%

Section: Comparison With Cpumentioning

confidence: 99%

“…Gawande et al 62 In summary, GPU provides higher raw performance and NVLink is crucial for GPU scaling. Also, KNL can provide performance/watt comparable to that of GPU.…”

Section: Comparison With Gpumentioning

confidence: 99%

See 2 more Smart Citations

A survey on evaluating and optimizing performance of Intel Xeon Phi

Mittal

2020

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…Along with novel TPU designs, GPU based solutions are still very common due to their intrinsic parallelism and numerous processing cores. The NVIDIA DGX [32] is one such system that attempts to accelerate deep learning.…”

Section: Cmos Based Neural Networkmentioning

confidence: 99%

Artificial neural networks based on memristive devices

Ravichandran

Banagozar

et al. 2018

Sci. China Inf. Sci.

View full text Add to dashboard Cite

The advent of memristive devices and the continuing research and development in the field of neuromorphic computing show great potential as an alternative to traditional von Neumann computing and bring us ever closer to realizing a true "thinking machine". Novel neural network architectures and algorithms inspired by the brain are becoming more and more attractive to keep up with computing needs, relying on intrinsic parallelism and reduced power consumption to outperform more conventional computing methods. This article provides an overview of various neural networks with an emphasis on networks based on memristive emerging devices, with the advantages of memristor neural networks compared with pure complementary metal oxide semiconductor (CMOS) implementations. A general description of neural networks is presented, followed by a survey of prominent CMOS networks, and finally networks implemented using emerging memristive devices are discussed, along with the motivation for developing memristor based networks and the computational potential these networks possess.

show abstract

Comparison of HPC Architectures for Computing All-Pairs Shortest Paths. Intel Xeon Phi KNL vs NVIDIA Pascal

Costanzo

Rucci

Costi

et al. 2021

Communications in Computer and Information Science

View full text Add to dashboard Cite

Today, one of the main challenges for high-performance computing systems is to improve their performance by keeping energy consumption at acceptable levels. In this context, a consolidated strategy consists of using accelerators such as GPUs or many-core Intel Xeon Phi processors. In this work, devices of the NVIDIA Pascal and Intel Xeon Phi Knights Landing architectures are described and compared. Selecting the Floyd-Warshall algorithm as a representative case of graph and memory-bound applications, optimized implementations were developed to analyze and compare performance and energy efficiency on both devices. As it was expected, Xeon Phi showed superior when considering double-precision data. However, contrary to what was considered in our preliminary analysis, it was found that the performance and energy efficiency of both devices were comparable using single-precision datatype.

show abstract

Scaling Deep Learning workloads: NVIDIA DGX-1/Pascal and Intel Knights Landing

Cited by 23 publications

References 13 publications

A survey on evaluating and optimizing performance of Intel Xeon Phi

A survey on evaluating and optimizing performance of Intel Xeon Phi

Artificial neural networks based on memristive devices

Comparison of HPC Architectures for Computing All-Pairs Shortest Paths. Intel Xeon Phi KNL vs NVIDIA Pascal

Contact Info

Product

Resources

About