Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysis

Reuther, Albert; Kepner, Jeremy; Byun, Chansup; Samsi, Siddharth; Arcand, William; Bestor, David; Bergeron, Bill; Gadepally, Vijay; Houle, Michael E.; Hubbell, Matthew; Jones, Michael; Klein, Anna; Milechin, Lauren; Mullen, Julia; Prout, Andrew; Rosa, Antonio; Yee, Charles; Michaleas, Peter

doi:10.1109/hpec.2018.8547629

Cited by 198 publications

(122 citation statements)

References 22 publications

Supporting

Mentioning

113

Contrasting

Order By: Relevance

“…This work overcomes obstacles to improved model accuracy by employing a novel hypersparse neural network analysis of "video" stream representations of the Internet traffic ( Figure 3). Utilizing recent innovations in interactive supercomputing [29], [30], matrix-based graph theory [ in fe re n c e i n f e r e n c e t r a i n i n g t r a i n i n g sparse image of internet traffic A t Fig. 3.…”

Section: Approachmentioning

confidence: 99%

Hypersparse Neural Network Analysis of Large-Scale Internet Traffic

Kepner

Cho²,

claffy³

et al. 2019

2019 IEEE High Performance Extreme Computing Conference (HPEC)

Self Cite

View full text Add to dashboard Cite

The Internet is transforming our society, necessitating a quantitative understanding of Internet traffic. Our team collects and curates the largest publicly available Internet traffic data containing 50 billion packets. Utilizing a novel hypersparse neural network analysis of "video" streams of this traffic using 10,000 processors in the MIT SuperCloud reveals a new phenomena: the importance of otherwise unseen leaf nodes and isolated links in Internet traffic. Our neural network approach further shows that a two-parameter modified Zipf-Mandelbrot distribution accurately describes a wide variety of source/destination statistics on moving sample windows ranging from 100,000 to 100,000,000 packets over collections that span years and continents. The inferred model parameters distinguish different network streams and the model leaf parameter strongly correlates with the fraction of the traffic in different underlying network topologies. The hypersparse neural network pipeline is highly adaptable and different network statistics and training models can be incorporated with simple changes to the image filter functions.

show abstract

Section: Approachmentioning

confidence: 99%

Hypersparse Neural Network Analysis of Large-Scale Internet Traffic

Kepner

Cho²,

claffy³

et al. 2019

2019 IEEE High Performance Extreme Computing Conference (HPEC)

Self Cite

View full text Add to dashboard Cite

show abstract

“…We describe the implementation of TapirXLA and the experimental setup to perform a fair comparison TapirXLA against XLA. We evaluated TapirXLA on a variety of multicore and manycore CPUs on the MIT Supercloud system [43], a heterogenous supercomputing system consisting of compute nodes with a variety of multicore and manycore processors.…”

Section: Discussionmentioning

confidence: 99%

“…We evaluated TapirXLA and XLA on all networks on a variety of multicore and manycore CPUs in the MIT Supercloud system [43], including an Intel Xeon Gold, an Intel Xeon E5, an Intel Xeon Phi, and an AMD Opteron. We followed the TensorFlow guidelines [48] and to set the threads used for intra-and inter-op thread counts equal to the number of processor cores and sockets on the system, respectively.…”

Section: Performance Comparison Of Tapirxla Versus Xlamentioning

confidence: 99%

TapirXLA: Embedding Fork-Join Parallelism into the XLA Compiler in TensorFlow Using Tapir

Schardl

Samsi

2019

2019 IEEE High Performance Extreme Computing Conference (HPEC)

Self Cite

View full text Add to dashboard Cite

This work introduces TapirXLA, a replacement for TensorFlow's XLA compiler that embeds recursive fork-join parallelism into XLA's low-level representation of code. Machinelearning applications rely on efficient parallel processing to achieve performance, and they employ a variety of technologies to improve performance, including compiler technology. But compilers in machine-learning frameworks lack a deep understanding of parallelism, causing them to lose performance by missing optimizations on parallel computation. This work studies how Tapir, a compiler intermediate representation (IR) that embeds parallelism into a mainstream compiler IR, can be incorporated into a compiler for machine learning to remedy this problem. TapirXLA modifies the XLA compiler in TensorFlow to employ the Tapir/LLVM compiler to optimize low-level parallel computation. TapirXLA encodes the parallelism within high-level TensorFlow operations using Tapir's representation of fork-join parallelism. TapirXLA also exposes to the compiler implementations of linear-algebra library routines whose parallel operations are encoded using Tapir's representation. We compared the performance of TensorFlow using TapirXLA against TensorFlow using an unmodified XLA compiler. On four neural-network benchmarks, TapirXLA speeds up the parallel running time of the network by a geometric-mean multiplicative factor of 30% to 100%, across four CPU architectures.

show abstract

“…The largest supercomputers currently available almost exclusively run the Linux operating system [8], [9]. Using these Linux powered supercomputers, it is possible to rapidly launch interactive applications on thousands of processors in a matter of seconds [10].…”

Section: Introductionmentioning

confidence: 99%

Interactive Launch of 16,000 Microsoft Windows Instances on a Supercomputer

Jones

Kepner

Orchard

et al. 2018

2018 IEEE High Performance Extreme Computing Conference (HPEC)

Self Cite

View full text Add to dashboard Cite

Simulation, machine learning, and data analysis require a wide range of software which can be dependent upon specific operating systems, such as Microsoft Windows. Running this software interactively on massively parallel supercomputers can present many challenges. Traditional methods of scaling Microsoft Windows applications to run on thousands of processors have typically relied on heavyweight virtual machines that can be inefficient and slow to launch on modern manycore processors. This paper describes a unique approach using the Lincoln Laboratory LLMapReduce technology in combination with the Wine Windows compatibility layer to rapidly and simultaneously launch and run Microsoft Windows applications on thousands of cores on a supercomputer. Specifically, this work demonstrates launching 16,000 Microsoft Windows applications in 5 minutes running on 16,000 processor cores. This capability significantly broadens the range of applications that can be run at large scale on a supercomputer.

show abstract

Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysis

Cited by 198 publications

References 22 publications

Hypersparse Neural Network Analysis of Large-Scale Internet Traffic

Hypersparse Neural Network Analysis of Large-Scale Internet Traffic

TapirXLA: Embedding Fork-Join Parallelism into the XLA Compiler in TensorFlow Using Tapir

Interactive Launch of 16,000 Microsoft Windows Instances on a Supercomputer

Contact Info

Product

Resources

About