Discrete Cosine Transform Spectral Pooling Layers for Convolutional Neural Networks

Smith, James Seale; Wilamowski, Bogdan M.

doi:10.1007/978-3-319-91253-0_23

Cited by 11 publications

(7 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In these experiments, spectral pooling shows favorable results. We also compare our DHT spectral pooling method with the DCT spectral pooling method [35]. Our implementation is based on PyTorch [28].…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Hartley Spectral Pooling for Deep Learning

Jianwei¹

2020

CSIAM-AM

View full text Add to dashboard Cite

In most convolution neural networks (CNNs), downsampling hidden layers is adopted for increasing computation efficiency and the receptive field size. Such operation is commonly called pooling. Maximization and averaging over sliding windows (max/average pooling), and plain downsampling in the form of strided convolution are popular pooling methods. Since the pooling is a lossy procedure, a motivation of our work is to design a new pooling approach for less lossy in the dimensionality reduction. Inspired by the spectral pooling proposed by Rippel et al. [1], we present the Hartley transform based spectral pooling method. The proposed spectral pooling avoids the use of complex arithmetic for frequency representation, in comparison with Fourier pooling. The new approach preserves more structure features for network's discriminability than max and average pooling. We empirically show the Hartley pooling gives rise to the convergence of training CNNs on MNIST and CIFAR-10 datasets.

show abstract

Section: Methodsmentioning

confidence: 99%

“…This would be tedious and sometimes computation/time consuming. Following the work of [1], spectral pooling approach using discrete cosine transform (DCT) [35] and wavelet transform [36] are also proposed.…”

Section: Introductionmentioning

confidence: 99%

Hartley Spectral Pooling for Deep Learning

Jianwei¹

2020

CSIAM-AM

View full text Add to dashboard Cite

show abstract

“…A flatten layer is implemented as a layer between the convolutional and dense layer to reduce the feature maps to a single one-dimensional vector. And the dense layer is a fully connected layer, similar to an MLP, which at a final stage of the CNN network, interprets the features extracted by the convolutional part of the model [3,37]. Figure 1 illustrates the one dimensional sequential CNN architecture, and how any input data is transformed by the convolutional operations into certain output.…”

Section: Convolutional Neural Networkmentioning

confidence: 99%

Time Series Forecasting on COVID-19 Data and Its Relevance to International Health Security

Kraamwinkel¹

2022

Contemporary Developments and Perspectives in International Health Security - Volume 3

View full text Add to dashboard Cite

The Corona virus pandemic is the most tragic virus outbreak in more than a century. Corona has globally already taken the lives of four million people, across all continents. The virus has the potential to become very catastrophic, if a significant part of the world population does not have any form of immunity against it. In this project, the aim is to make forecasts on the number of daily infections in the Netherlands. Seven different models were implemented to forecast the number of infected people in a three-month time period. The sequential CNN model outperformed all other models substantially. The capabilities of CNN models in time series forecasting can be very encouraging in conducting more research on time series data with convolutional neural networks.

show abstract

“…The Discrete Cosine Transform (DCT) is also a frequency domain transform which is being utilized in pooling layers [16,17,18]. These papers propose modifications to preserve more information or reduce the layer computational cost.…”

Section: Dct-based Poolingmentioning

confidence: 99%

Image classification in frequency domain with 2SReLU: a second harmonics superposition activation function

Watanabe¹,

Wolf²

2020

Preprint

View full text Add to dashboard Cite

Deep Convolutional Neural Networks are able to identify complex patterns and perform tasks with super-human capabilities. However, besides the exceptional results, they are not completely understood and it is still impractical to hand-engineer similar solutions. In this work, an image classification Convolutional Neural Network and its building blocks are described from a frequency domain perspective. Some network layers have established counterparts in the frequency domain like the convolutional and pooling layers. We propose the 2SReLU layer, a novel non-linear activation function that preserves high frequency components in deep networks. It is demonstrated that in the frequency domain it is possible to achieve competitive results without using the computationally costly convolution operation. A source code implementation in PyTorch is provided at: https: //gitlab.com/thomio/2srelu.

show abstract

Discrete Cosine Transform Spectral Pooling Layers for Convolutional Neural Networks

Cited by 11 publications

References 7 publications

Hartley Spectral Pooling for Deep Learning

Hartley Spectral Pooling for Deep Learning

Time Series Forecasting on COVID-19 Data and Its Relevance to International Health Security

Image classification in frequency domain with 2SReLU: a second harmonics superposition activation function

Contact Info

Product

Resources

About