Fast Walsh-Hadamard Transform and Smooth-Thresholding Based Binary Layers in Deep Neural Networks

Pan, Hongyi; Badawi, Diaa; Çetin, A. Enis

doi:10.1109/cvprw53098.2021.00523

Cited by 15 publications

(3 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is described by the following binary matrix:

, a

22

WH transform Y of the vector

. In general, the WH transform

of a vector

where

can be expressed via the orthogonal Walsh matrix

which is generated using the Hadamard matrix and can be recursively constructed [40] .…”

Section: Methodsmentioning

confidence: 99%

“…Previously, WH transform has been used in signal processing to reduce the complexity of electronic signal [41] , image processing [40] to replace complex layers with a simpler version of WH, and it has been used in Genomics as well [42] in determining different diseases based on DNA and RNA sequencing. These inputs (signals, DNA, RNA) are sequential data similar to text data.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Using transfer learning-based causality extraction to mine latent factors for Sjögren's syndrome from biomedical literature

VanSchaik,

Jain,

Rajapuri

et al. 2023

Heliyon

View full text Add to dashboard Cite

“…It is described by the following binary matrix:

, a

22

WH transform Y of the vector

. In general, the WH transform

of a vector

where

can be expressed via the orthogonal Walsh matrix

which is generated using the Hadamard matrix and can be recursively constructed [40] .…”

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Using transfer learning-based causality extraction to mine latent factors for Sjögren's syndrome from biomedical literature

VanSchaik,

Jain,

Rajapuri

et al. 2023

Heliyon

View full text Add to dashboard Cite

“…Walsh-Hadamard transform (WHT) [17], [18], [19], DCT [20], [21], [22], block circulant matrices (BCMs) [23], DFT [24], discrete wavelet transform (DWT) [25], singular value decomposition (SVD) [26], and principal component analysis (PCA) [27].…”

Section: Introductionmentioning

confidence: 99%

ADC/DAC-Free Analog Acceleration of Deep Neural Networks With Frequency Transformation

Darabi,

Hashem,

Pan

et al. 2024

IEEE Trans. VLSI Syst.

Self Cite

View full text Add to dashboard Cite

The edge processing of deep neural networks (DNNs) is becoming increasingly important due to its ability to extract valuable information directly at the data source to minimize latency and energy consumption. Although pruning techniques are commonly used to reduce model size for edge computing, they have certain limitations. Frequency-domain model compression, such as with the Walsh-Hadamard transform (WHT), has been identified as an efficient alternative. However, the benefits of frequency-domain processing are often offset by the increased multiply-accumulate (MAC) operations required. This article proposes a novel approach to an energy-efficient acceleration of frequency-domain neural networks by utilizing analog-domain frequency-based tensor transformations. Our approach offers unique opportunities to enhance computational efficiency, resulting in several high-level advantages, including array microarchitecture with parallelism, analog-to-digital converter (ADC)/digital-to-analog converter (DAC)-free analog computations, and increased output sparsity. Our approach achieves more compact cells by eliminating the need for trainable parameters in the transformation matrix. Moreover, our novel array microarchitecture enables adaptive stitching of cells column-wise and row-wise, thereby facilitating perfect parallelism in computations. Additionally, our scheme enables ADC/DAC-free computations by training against highly quantized matrix-vector products, leveraging the parameter-free nature of matrix multiplications. Another crucial aspect of our design is its ability to handle signed-bit processing for frequencybased transformations. This leads to increased output sparsity and reduced digitization workload. On a 16 × 16 crossbars, for 8-bit input processing, the proposed approach achieves the energy efficiency of 801 tera operations per second per Watt (TOPS/W) without early termination strategy and 2655 TOPS/W with early termination strategy at VDD = 0.85 V for 16-nm predictive technology models (PTM).

show abstract

Energy-Efficient Image Processing Using Binary Neural Networks with Hadamard Transform

Park

Lee

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Fast Walsh-Hadamard Transform and Smooth-Thresholding Based Binary Layers in Deep Neural Networks

Cited by 15 publications

References 35 publications

Using transfer learning-based causality extraction to mine latent factors for Sjögren's syndrome from biomedical literature

Using transfer learning-based causality extraction to mine latent factors for Sjögren's syndrome from biomedical literature

ADC/DAC-Free Analog Acceleration of Deep Neural Networks With Frequency Transformation

Energy-Efficient Image Processing Using Binary Neural Networks with Hadamard Transform

Contact Info

Product

Resources

About