YaConv: Convolution with Low Cache Footprint

Abstract

This paper introduces YaConv , a new algorithm to compute convolution using GEMM microkernels from a BLAS library that is efficient for multiple CPU architectures. Previous approaches either create a copy of each image element for each filter element or reload these elements into cache for each GEMM call, leading to redundant instances of the image elements in cache. Instead, YaConv loads each image element once into the cache and maximizes the reuse of these elements. The output image is computed by scattering results of the GEMM microkernel calls to the correct locations in the output image. The main advantage of this new algorithm — which leads to better performance in comparison to the existing im2col approach on several architectures — is a more efficient use of the memory hierarchy. The experimental evaluation on convolutional layers from PyTorch, along with a parameterized study, indicates an average 24% speedup over im2col convolution. Increased performance comes as a result of 3 × reduction in L3 cache accesses and 2 × fewer branch instructions.

Advancing Direct Convolution using Convolution Slicing Optimization and ISA Extensions

“…Korostelev et al [17] combine the ideas of modifying the GEMM routine and decomposing convolution into multiple GEMM operations [2] to create a new method that avoids packing redundancy while keeping the overall GEMM structure (tiling, packing, micro-kernel). Contrary to the approach in SConv, [17] works only on unitary stride convolutions and improves only non-pointwise convolutions. Moreover, their experiments focus on isolated convolutions and do not provide results for complete models.…”