Dynamic allocation for scratch-pad memory using compile-time decisions

Abstract

In this research we propose a highly predictable, low overhead and yet dynamic, memory allocation strategy for embedded systems with scratch-pad memory. A scratch-pad is a fast compiler-managed SRAM memory that replaces the hardware-managed cache. It is motivated by its better real-time guarantees vs cache and by its significantly lower overheads in energy consumption, area and overall runtime, even with a simple allocation scheme.Scratch-pad allocation primarily methods are of two types. First, software-caching schemes emulate the workings of a hardware cache in software. Instructions are inserted before each load/store to check the softwaremaintained cache tags. Such methods incur large overheads in runtime, code size, energy consumption and SRAM space for tags and deliver poor real-time guarantees just like hardware caches. A second category of algorithms partitions variables at compile-time into the two banks. However, a drawback of such static allocation schemes is that they do not account for dynamic program behavior. It is easy to see why a data allocation that never changes at runtime cannot achieve the full locality benefits of a cache.We propose a dynamic allocation methodology for global and stack data and program code that, (i) accounts for changing program requirements at runtime (ii) has no software-caching tags (iii) requires no run-time checks (iv) has extremely low overheads, and (v) yields 100% predictable memory access times. In this method data that is about to be accessed frequently is copied into the scratch-pad using compiler-inserted code at fixed and infrequent points in the program. Earlier data is evicted if necessary. When compared to a provably optimal static allocation, results show that our scheme reduces runtime by up to 39.8% and energy by up to 31.3% on average for our benchmarks, depending on the SRAM size used. The actual gain depends on the SRAM size, but our results show that close to the maximum benefit in run-time and energy is achieved for a substantial range of small SRAM sizes commonly found in embedded systems. Our comparison with a direct mapped cache shows that our method performs roughly as well as a cached architecture.

Practical Loop Transformations for Tensor Contraction Expressions on Multi-level Memory Hierarchies

“…Li et al proposed interval coloring approach for arranging scratchpad memory, utilizing the observation that live ranges of two arrays should not interfere or one should not contain the other [20]. Udayakumaran et al address the scratch-pad memory allocation in a dynamic way, using a greedy heuristic as the cost model [37]. None of these efforts have considered multi-level data movement.…”

Optimizing Local Memory Allocation and Assignment through a Decoupled Approach

“…Some heuristics exist [16,17,18,19] but little is known about the optimization problem, its complexity and the interplay with other optimizations. The burning hot question is of course: does the decoupled approach hold for the LM management problem?…”

“…It paritions the LM for each different array size, performs live range spliting and use a register allocation framework to perform memory coloring. The second approach [16,17,24] allocates data onto the scratch-pad memory between program regions separated by specific program points. More specifically, allocation is based on the access frequency-per-byte of a variable in a region (collected from profile data).…”

“…Most numerical and signal-processing codes exhibit a vast majority of dense, uniform access patterns. In such cases, homogenization is trivial: it suffices to set the same strip-mining factor for all loops operating over a given array [17]. The last step is to abstract the transformed code, isolating array block operations into atomic regions.…”

“…When generating the code for a real target, one needs to insert code for array block load/store, and pointer moves. Again, standard techniques exist when array accesses are uniform [17,18].…”