Memory-Efficient CNN Accelerator Based on Interlayer Feature Map Compression

Shao, Zhuang; Chen, Xiaoliang; Du, Li; Chen, Lei; Du, Yuan; Zhuang, Wei; Wei, Huadong; Xie, Chenjia; Wang, Zhongfeng

doi:10.1109/tcsi.2021.3120312

Cited by 20 publications

(9 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This work analyzed the compression ratio of each fused layer and provided the overall compression ratio of each CNN. A fused layer is a convolution layer, a batch norm layer, an activation layer, and a pooling layer if the network has [39]. The results proved that our method can not only reduce the number of data interactions to on-chip storage and off-chip memory but also can effectively reduce the required on-chip storage area while ensuring that the average accuracy loss is around 0.6%.…”

Section: Experiments Resultsmentioning

confidence: 85%

“…In Table III, the proposed method is compared with some state-of-the-art compression methods. This work has a better compression ratio and network accuracy in VGG-16, ResNet-50, and MobileNet-v2 compared to the methods proposed in [15,39]. In terms of hardware implementation, the hardware reported in [15] integrates the entire process of PCA, resulting large area overhead.…”

Section: ) Compact Networkmentioning

confidence: 99%

“…For the regular convolution layers, the proposed method can still effectively * The Overall compression ratio of the interlayer feature map. N/A corresponding to no feature map compression implemented; 62.5% and 38.02% corresponding to the compression ratios of JSSC'16 [34] and JSSC'20 [30] on AlexNet; 30.63%~71.05% corresponding to the compression ratios of TCASI'21 [39] on VGG-16-BN, ResNet-50, MobileNet-v1, MobileNet-v2, AlexNet and Yolo-v3. 9.8%-19.3% corresponding to the compression ratios of this work of VGG-16, ResNet-18, ResNet-50, MobileNet-v2, MobileNet-v3-small, and Inception-v3 in 16bit quantized mode.…”

Section: ) Compact Networkmentioning

confidence: 99%

“…Considering this, some compression methods that do not rely on sparsity are proposed. In [39], the Discrete Cosine Transform (DCT) is utilized to compress the interlayer feature maps. Still, it only worked well in the front layers of CNNs, where the feature maps have similar object shapes to the original images, and only the high-frequency components of the intra-channel data are removed.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

An Efficient CNN Inference Accelerator Based on Intra- and Inter-Channel Feature Map Compression

Xie¹,

Shao²,

Zhao³

et al. 2022

Preprint

View full text Add to dashboard Cite

<p>Deep convolutional neural networks (CNNs) generate intensive inter-layer data during inference, which results in substantial on-chip memory size and off-chip bandwidth. To solve the memory constraint, this paper proposes an accelerator adopted with a compression technique that can reduce the inter-layer data by removing both intra- and inter-channel redundant information. Principal component analysis (PCA) is utilized in the compression process to concentrate inter-channel information. The spatial differences, truncation, and reconfigurable bit-width coding are implemented inside every feature map to eliminate the intra-channel data redundancy. Moreover, a particular data arrangement is introduced to enhance data continuity to optimize PCA analysis and improve compression performance. A CNN accelerator with the proposed compression technique is designed to support the on-the-fly compression process by pipelining the reconstruction, CNN computation, and compression operation. The prototype accelerator is implemented using 28-nm CMOS technology. It achieves 819.2GOPS peak throughput and 3.75TOPS/W energy efficiency with 218.5mW. Experiments show that the proposed compression technique achieves a compression ratio of 21.5%~43.0% (8-bit mode) and 9.8%~19.3% (16-bit mode) on state-of-the-art CNNs with a negligible accuracy loss. </p>

show abstract

Section: Experiments Resultsmentioning

confidence: 85%

Section: ) Compact Networkmentioning

confidence: 99%

Section: ) Compact Networkmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An Efficient CNN Inference Accelerator Based on Intra- and Inter-Channel Feature Map Compression

Xie¹,

Shao²,

Zhao³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For a CNN-based SE algorithm to be suitably implemented in memory, power, and computation speed-constrained systems such as hearing devices, significant model compression and acceleration are required. Approaches for CNN compression can be divided into three categories: network pruning [7], precision reduction [8] and design of compact network architectures [9].…”

Section: Introductionmentioning

confidence: 99%

A Compact CNN-Based Speech Enhancement With Adaptive Filter Design Using Gabor Function and Region-Aware Convolution

2022

View full text Add to dashboard Cite

Speech enhancement (SE) is used in many applications, such as hearing devices, to improve speech intelligibility and quality. Convolutional neural network-based (CNN-based) SE algorithms in literature often employ generic convolutional filters that are not optimized for SE applications. This paper presents a CNN-based SE algorithm with an adaptive filter design (named 'CNN-AFD') using Gabor function and region-aware convolution. The proposed algorithm incorporates fixed Gabor functions into convolutional filters to model human auditory processing for improved denoising performance. The feature maps obtained from the Gabor-incorporated convolutional layers serve as learnable guided masks (tuned at backpropagation) for generating adaptive custom region-aware filters. The custom filters extract features from speech regions (i.e., 'region-aware') while maintaining translation-invariance. To reduce the high cost of inference of the CNN, skip convolution and activation analysis-wise pruning are explored. Employing skip convolution allowed the training time per epoch to be reduced by close to 40%. Pruning of neurons with high numbers of zero activations complements skip convolution and significantly reduces model parameters by more than 30%. The proposed CNN-AFD outperformed all four CNN-based SE baseline algorithms (i.e., a CNN-based SE employing generic filters, a CNN-based SE without region-aware convolution, a CNN-based SE trained with complex spectrograms and a CNN-based SE processing in the time-domain) with an average of 0.95, 1.82 and 0.82 in short-time objective intelligibility (STOI), perceptual evaluation of speech quality (PESQ) and logarithmic spectral distance (LSD) scores, respectively, when tasked to denoise speech contaminated with NOISEX-92 noises at -5, 0 and 5 dB signal-to-noise ratios (SNRs).

show abstract

Feature Map Compression for Video Coding for Machines Based on Receptive Block Based Principal Component Analysis

et al. 2023

View full text Add to dashboard Cite

Memory-Efficient CNN Accelerator Based on Interlayer Feature Map Compression

Cited by 20 publications

References 34 publications

An Efficient CNN Inference Accelerator Based on Intra- and Inter-Channel Feature Map Compression

An Efficient CNN Inference Accelerator Based on Intra- and Inter-Channel Feature Map Compression

A Compact CNN-Based Speech Enhancement With Adaptive Filter Design Using Gabor Function and Region-Aware Convolution

Feature Map Compression for Video Coding for Machines Based on Receptive Block Based Principal Component Analysis

Contact Info

Product

Resources

About