Eliminating redundant fragment shader executions on a mobile GPU via hardware memoization

Abstract

Redundancy is at the heart of graphical applications. In fact, generating an animation typically involves the succession of extremely similar images. In terms of rendering these images, this behavior translates into the creation of many fragment programs with the exact same input data. We have measured this fragment redundancy for a set of commercial Android applications, and found that more than 40% of the fragments used in a frame have been already computed in a prior frame. In this paper we try to exploit this redundancy, using fragment memoization. Unfortunately, this is not an easy task as most of the redundancy exists across frames, rendering most HW based schemes unfeasible. We thus first take a step back and try to analyze the temporal locality of the redundant fragments, their complexity, and the number of inputs typically seen in fragment programs. The result of our analysis is a task level memoization scheme, that easily outperforms the current state-of-the-art in low power GPUs More specifically, our experimental results show that our scheme is able to remove 59.7% of the redundant fragment computations on average. This materializes to a significant speedup of 17.6% on average, while also improving the overall energy efficiency by 8.9% on average.

A case for core-assisted bottleneck acceleration in GPUs

“…We envision multiple applications for the CABA framework, e.g., data compression [4,22,65,84], memoization [12,26,77], data prefetching [13,34,47,64]. In Section 4, we provide a detailed case study of enabling data compression with the framework, discussing various tradeo s. We believe CABA can be useful for many other optimizations, and we discuss some of them brie y in Section 7.…”

“…Prior work [8,12,70] observed redundancy in inputs to data in GPU workloads. In applications limited by available compute resources, memoization o ers an opportunity to trade o computation for storage, thereby enabling potentially higher energy e ciency and performance.…”

“…In order to realize memoization in hardware, a look-up table (LUT) is required to dynamically cache the results of computations as well as the corresponding inputs. The granularity of computational reuse can be at the level of fragments [12], basic blocks, functions [7,9,26,40,77], or longlatency instructions [23]. The CABA framework is a natural way to implement such an optimization.…”

“…The generality of CABA enables its use in alleviating other bottlenecks with di erent optimizations. We discuss two examples: (1) using assist warps to perform memoization to eliminate redundant computations that have the same or similar inputs [12,26,77], by storing the results of frequently-performed computations in the main memory hierarchy (i.e., by converting the computational problem into a storage problem) and, (2) using the idle memory pipeline to perform opportunistic prefetching to better overlap computation with memory access. Assist warps o er a hardware/software interface to implement hybrid prefetching algorithms [31] with varying degrees of complexity.…”

On‐the‐Fly Power‐Aware Rendering

“…To extend the limited battery life, energy saving becomes a primary goal [AMS08,JGDAM12]. A lot of research effort has recently been oriented towards characterising the power consumption of rendering algorithms and finding strategies to control the amount of expended energy [SPP*15,PLS11,APX14,WYM*16].…”