CosmoFlow: Using Deep Learning to Learn the Universe at Scale

Mathuriya, Amrita; Bard, Deborah; Mendygral, Peter; Meadows, Lawrence; Arnemann, James; Shao, Lei; He, Siyu; Kärnä, Tuomas; Moise, Diana; Pennycook, John; Maschhoff, Kristyn J.; Sewall, Jason; Kumar, Nalini; Ho, Shirley; Ringenburg, Michael F.; Prabhat, Prabhat; Lee, Victor

doi:10.1109/sc.2018.00068

Cited by 92 publications

(68 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[32]- [34]). These works have primarily focused on scaling sample-parallel training via optimizing communication and I/O, large mini-batches, etc.…”

Section: Related Workmentioning

confidence: 99%

Improving Strong-Scaling of CNN Training by Exploiting Finer-Grained Parallelism

Dryden

Maruyama

Benson

et al. 2019

2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS)

View full text Add to dashboard Cite

B. Performance modelingWe make use of analytic models for some of our performance estimates, particularly communication. We use a linear model [21] for communication, where α is the latency and β is the inverse bandwidth. Then the cost to send a message between two nodes is α + βn. We additionally assume that the network is full-duplex and that there is no interference.Collective communication operations such as allreduce will be important for some operations; for these, we use the performance models of Thakur et al [22]. For distributed matrix multiplication, we use the performance models developed for the Elemental library [23]. C. NotationWe now define some notation for distributed tensors that will be used throughout this paper. Our notation is heavily based on the tensor notation developed for the FLAME project [23]- [25].A tensor is an M -dimensional array, where the size of dimension m is I m , and we write I = (I 0 , . . . , I M −1 ) to refer to the shape of an entire tensor.

show abstract

“…[32]- [34]). These works have primarily focused on scaling sample-parallel training via optimizing communication and I/O, large mini-batches, etc.…”

Section: Related Workmentioning

confidence: 99%

Improving Strong-Scaling of CNN Training by Exploiting Finer-Grained Parallelism

Dryden

Maruyama

Benson

et al. 2019

2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS)

View full text Add to dashboard Cite

show abstract

“…For solving this problem, Ravanbakhsh et al (2017) [24] firstly propose a 3D CNN model with 6 convolutional layers and 3 fully-connected layers and opens the way to estimating the parameters with high accuracy. Mathuriya et al (2018) propose CosmoFlow [21], which is a project aiming to process large 3D cosmology dataset on HPC systems. They extend the CNN model designed by Ravanbakhsh et al (2017) [24].…”

Section: Cosmologymentioning

confidence: 99%

HPC AI500: A Benchmark Suite for HPC AI Systems

Jiang

Gao

Wang

et al. 2019

Benchmarking, Measuring, and Optimizing

View full text Add to dashboard Cite

In recent years, with the trend of applying deep learning (DL) in high performance scientific computing, the unique characteristics of emerging DL workloads in HPC raise great challenges in designing, implementing HPC AI systems. The community needs a new yard stick for evaluating the future HPC systems. In this paper, we propose HPC AI500 -a benchmark suite for evaluating HPC systems that running scientific DL workloads. Covering the most representative scientific fields, each workload from HPC AI500 is based on real-world scientific DL applications. Currently, we choose 14 scientific DL benchmarks from perspectives of application scenarios, data sets, and software stack. We propose a set of metrics for comprehensively evaluating the HPC AI systems, considering both accuracy, performance as well as power and cost. We provide a scalable reference implementation of HPC AI500. The specification and source code are publicly available from http://www.benchcouncil.org/HPCAI500/index.html. Meanwhile, the AI benchmark suites for datacenter, IoT, Edge are also released on the BenchCouncil web site.

show abstract

“…Richard et al [34] apply a neural network to ITER magnets to predict the occurrence of the interruption, which can be used to adjust the reaction to continue generating power and avoid ITER damage. Mathuriya et al [36] build a CNN model to determine the physical model that describes our universe.…”

Section: Related Workmentioning

confidence: 99%

Adaptive neural network-based approximation to accelerate eulerian fluid simulation

Dong

Liu

Xie

et al. 2019

Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

The Eulerian fluid simulation is an important HPC application. The neural network has been applied to accelerate it. The current methods that accelerate the fluid simulation with neural networks lack flexibility and generalization. In this paper, we tackle the above limitation and aim to enhance the applicability of neural networks in the Eulerian fluid simulation. We introduce Smart-fluidnet, a framework that automates model generation and application. Given an existing neural network as input, Smart-fluidnet generates multiple neural networks before the simulation to meet the execution time and simulation quality requirement. During the simulation, Smartfluidnet dynamically switches the neural networks to make best efforts to reach the user's requirement on simulation quality. Evaluating with 20,480 input problems, we show that Smart-fluidnet achieves 1.46x and 590x speedup comparing with a state-of-the-art neural network model and the original fluid simulation respectively on an NVIDIA Titan X Pascal GPU, while providing better simulation quality than the state-of-the-art model. CCS CONCEPTS • Computing methodologies → Parallel algorithms; Neural networks; Model development and analysis.

show abstract

CosmoFlow: Using Deep Learning to Learn the Universe at Scale

Cited by 92 publications

References 19 publications

Improving Strong-Scaling of CNN Training by Exploiting Finer-Grained Parallelism

Improving Strong-Scaling of CNN Training by Exploiting Finer-Grained Parallelism

HPC AI500: A Benchmark Suite for HPC AI Systems

Adaptive neural network-based approximation to accelerate eulerian fluid simulation

Contact Info

Product

Resources

About