Fast Computational GPU Design with GT-Pin

Kambadur, Melanie; Hong, Sunpyo; Cabral, J.J.S.P.; Patil, Harish; Luk, Chi-Keung; Sajid, Sohaib; Kim, Martha A.

doi:10.1109/iiswc.2015.14

Cited by 28 publications

(11 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…GPU simulators [8,22,43] monitor execution details, but incur prohibitively high overhead for real usage. To reduce the measurement overhead, one can instrument GPU binaries with SASSI [46], NVBit [48], and GTPin [21], or bytecode using LLVM [27]. However, these instrumentation engines do not directly identify value-related inefficiencies.…”

Section: Related Workmentioning

confidence: 99%

ValueExpert: exploring value patterns in GPU-accelerated applications

Zhou

Hao

Mellor-Crummey

et al. 2022

Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

View full text Add to dashboard Cite

General-purpose GPUs have become common in modern computing systems to accelerate applications in many domains, including machine learning, high-performance computing, and autonomous driving. However, inefficiencies abound in GPU-accelerated applications, which prevent them from obtaining bare-metal performance. Performance tools play an important role in understanding performance inefficiencies in complex code bases. Many GPU performance tools pinpoint time-consuming code and provide high-level performance insights but overlook one important performance issueÐvalue-related inefficiencies, which exist in many GPU code bases. In this paper, we present ValueExpert, a novel tool to pinpoint value-related inefficiencies in GPU applications.ValueExpert monitors application execution to capture values produced and used by each load and store operation in GPU kernels, recognizes multiple value patterns, and provides intuitive optimization guidance. We address systemic challenges in collecting, maintaining, and analyzing voluminous performance data from many GPU threads to make ValueExpert applicable to complex applications. We evaluate ValueExpert on a wide range of welltuned benchmarks and applications, including PyTorch, Darknet, LAMMPS, Castro, and many others. ValueExpert is able to identify previously unknown performance issues and provide suggestions for nontrivial performance improvements with typically less than five lines of code changes. We verify our optimizations with application developers and upstream fixes to their repositories.

show abstract

Section: Related Workmentioning

confidence: 99%

ValueExpert: exploring value patterns in GPU-accelerated applications

Zhou

Hao

Mellor-Crummey

et al. 2022

Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

View full text Add to dashboard Cite

show abstract

“…Simulating an entire GPU application on a cycle-level simulator [32], [33] is often impractical and this is even more true for long-running SQNN training applications. To aid in successful simulation of long-running applications, prior works have attempted to identify representative regions within applications and porting them to simulators for CPUs [4], [18], [34]- [36] and GPUs [37], [38].…”

Section: A Enabling Network-level Simulation For Sqnnsmentioning

confidence: 99%

SeqPoint: Identifying Representative Iterations of Sequence-Based Neural Networks

Pati

Aga

Sinclair

et al. 2020

2020 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS)

View full text Add to dashboard Cite

The ubiquity of deep neural networks (DNNs) continues to rise, making them a crucial application class for hardware optimizations. However, detailed profiling and characterization of DNN training remains difficult as these applications often run for hours to days on real hardware. Prior works have exploited the iterative nature of DNNs to profile a few training iterations to represent the entire training run. While such a strategy is sound for networks like convolutional neural networks (CNNs), where the nature of the computation is largely input independent, we observe in this work that this approach is sub-optimal for sequence-based neural networks (SQNNs) such as recurrent neural networks (RNNs). The amount and nature of computations in SQNNs can vary for each input, resulting in heterogeneity across iterations. Thus, arbitrarily selecting a few iterations is insufficient to accurately summarize the behavior of the entire training run. To tackle this challenge, we carefully study the factors that impact SQNN training iterations and identify input sequence length as the key determining factor for variations across iterations. We then use this observation to characterize all iterations of an SQNN training run (requiring no profiling or simulation of the application) and select representative iterations, which we term SeqPoints. We analyze two state-ofthe-art SQNNs, DeepSpeech2 and Google's Neural Machine Translation (GNMT), and show that SeqPoints can represent their entire training runs accurately, resulting in geomean errors of only 0.11% and 0.53%, respectively, when projecting overall runtime and 0.13% and 1.50% when projecting speedups due to architectural changes. This high accuracy is achieved while reducing the time needed for profiling by 345x and 214x for the two networks compared to full training runs. As a result, SeqPoint can enable analysis of SQNN training runs in mere minutes instead of hours or days.

show abstract

“…GPU vendors have also developed instrumentation tools [22,26,36,37] for fine-grained performance measurement and analysis. These tools, however, introduce unavoidable overhead for GPU kernels.…”

Section: Related Workmentioning

confidence: 99%

GPA: A GPU Performance Advisor Based on Instruction Sampling

Zhou

Meng

Sai

et al. 2020

Preprint

View full text Add to dashboard Cite

Developing efficient GPU kernels can be difficult because of the complexity of GPU architectures and programming models. Existing performance tools only provide coarse-grained suggestions at the kernel level, if any. In this paper, we describe GPA, a performance advisor for NVIDIA GPUs that suggests potential code optimization opportunities at a hierarchy of levels, including individual lines, loops, and functions. To relieve users of the burden of interpreting performance counters and analyzing bottlenecks, GPA uses data flow analysis to approximately attribute measured instruction stalls to their root causes and uses information about a program's structure and the GPU to match inefficiency patterns with suggestions for optimization. To quantify each suggestion's potential benefits, we developed PC samplingbased performance models to estimate its speedup. Our experiments with benchmarks and applications show that GPA provides an insightful report to guide performance optimization. Using GPA, we obtained speedups on a Volta V100 GPU ranging from 1.03× to 3.86×, with a geometric mean of 1.22×.

show abstract

Fast Computational GPU Design with GT-Pin

Cited by 28 publications

References 18 publications

ValueExpert: exploring value patterns in GPU-accelerated applications

ValueExpert: exploring value patterns in GPU-accelerated applications

SeqPoint: Identifying Representative Iterations of Sequence-Based Neural Networks

GPA: A GPU Performance Advisor Based on Instruction Sampling

Contact Info

Product

Resources

About