A New Global Pooling Method for Deep Neural Networks: Global Average of Top-K Max-Pooling

Dogan, Yahya

doi:10.18280/ts.400216

Cited by 9 publications

(4 citation statements)

References 32 publications

(65 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In value-based pooling methods [10][11][12][14][15][16], a value selection is determined based on a criterion among the pooling region's values. Mixed pooling [10] adds a parameter to choose between maximum and average pooling.…”

Section: Related Workmentioning

confidence: 99%

Which pooling method is better: Max, Avg, or Concat (Max, Avg)

Doğan

2024

Communications Faculty of Sciences University of Ankara Series A2-A3 Physical Sciences and Engineering

View full text Add to dashboard Cite

Pooling is a non-linear operation that aggregates the results of a given region to a single value. This method effectively removes extraneous details in feature maps while keeping the overall information. As a result, the size of feature maps is reduced, which decreases computing costs and prevents overfitting by eliminating irrelevant data. In CNN models, the max pooling and average pooling methods are commonly utilized. The max pooling selects the highest value within the pooling area and aids in preserving essential features of the image. However, it ignores the other values inside the pooling region, resulting in a significant loss of information. The average pooling computes the average values within the pooling area, which reduces data loss. However, by failing to emphasize critical pixels in the image, it may result in the loss of significant features. To examine the performance of pooling methods, this study comprised the experimental analysis of multiple models, i.e. shallow and deep, datasets, i.e. Cifar10, Cifar100, and SVHN, and pool sizes, e.g. $2x2$, $3x3$, $10x10$. Furthermore, the study investigated the effectiveness of combining two approaches, namely Concat (Max, Avg), to minimize information loss. The findings of this work provide an important guideline for selecting pooling methods in the design of CNNs. The experimental results demonstrate that pooling methods have a considerable impact on model performance. Moreover, there are variances based on the model and pool size.

show abstract

Section: Related Workmentioning

confidence: 99%

Which pooling method is better: Max, Avg, or Concat (Max, Avg)

Doğan

2024

Communications Faculty of Sciences University of Ankara Series A2-A3 Physical Sciences and Engineering

View full text Add to dashboard Cite

show abstract

“…This operation captures the essential feature information from each filter by selecting the top-K maximum values. The purpose of the K-Max pooling operation is to extract the most representative features, allowing the model to understand better the input data's important aspects [23]. In this manner, the most significant features are retained while reducing irrelevant information, thus enhancing the model's performance and accuracy in processing sequence data.…”

Section: The Convolutional Neural Network Layermentioning

confidence: 99%

Electric Load Forecasting Based on Deep Ensemble Learning

Wang,

Yu,

Wang

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

Short-to-medium-term electric load forecasting is crucial for grid planning, transformation, and load scheduling for power supply departments. Various complex and ever-changing factors such as weather, seasons, regional economic structures, and enterprise production cycles exert uncontrollable effects on the electric grid load. While the causal convolutional neural network can significantly enhance long-term sequence prediction, it may suffer from problems such as vanishing gradients and overfitting due to extended time series. To address this issue, this paper introduces a new power load data anomaly detection method, which leverages a convolutional neural network (CNN) to extract temporal and spatial information from the load data. The features extracted are then processed using a bidirectional long short-term memory network (BiLSTM) to capture the temporal dependencies in the data more adeptly. An enhanced random forest (RF) classifier is employed for anomaly detection in electric load data. Furthermore, the paper proposes a new model framework for electricity load forecasting that combines a dilated causal convolutional neural network with ensemble learning. This combination addresses issues such as vanishing gradients encountered in causal convolutional neural networks with long time series. Extreme gradient boosting (XGBoost), category boosting (CATBoost), and light gradient boosting machine (LightGBM) models act as the base learners for ensemble modeling to comprehend deep cross-features, and the prediction results generated by ensemble learning serve as a new feature set for secondary ensemble modeling. The dilated convolutional neural network broadens the receptive field of the convolutional kernel. All acquired feature values are concatenated and input into the dilated causal convolutional neural network for training, achieving short-to-medium-term electric load forecasting. Experimental results indicate that compared to existing models, its root mean squared error (RMSE) and mean squared error (MSE) in short-term and mid-term electricity load forecasting are reduced by 4.96% and 12.31%, respectively, underscoring the efficacy of the proposed framework.

show abstract

“…Using data from satellite photography, this research studies deep learning algorithms to identify deforested areas in forests and optimize these detections to yield reliable results. With a wide range of possible applications in fields including food and health, deep learning stands out as a significant technological advancement [6][7][8]. To successfully complete the segmentation task and deliver advanced results, many machine learning techniques can be applied [9].…”

Section: Introductionmentioning

confidence: 99%

Accurate Detection of Treeless Areas in Forests Using Deep Learning Models

2023

Proceeding Book of 2nd International Conference on Recent Academic Studies ICRAS 2023

View full text Add to dashboard Cite

In order to maintain biological diversity, control climate, and manage sustainable resources, forests are an essential part of the global ecosystem. To improve the efficacy of forestry methods, estimate the effects of natural disasters, and fortify environmental legislation, it is crucial to monitor and classify forests. This study applies deep learning models, especially U-Net and Res-Net, to imagery from satellites to identify deforested areas within forests. The models were studied for segmentation, and the resulting data was compared. With 96.1% accuracy, 27% IoU, and 92.7% Dice Coefficient, the U-Net model performed quite well in identifying deforested areas inside of forests. As a result of these excellent results, U-Net architecture-based applications have the potential to positively impact a wide range of applications, from forestry to monitoring and predicting fires, assessing the consequences of natural disasters, and improving environmental policy.

show abstract

A New Global Pooling Method for Deep Neural Networks: Global Average of Top-K Max-Pooling

Cited by 9 publications

References 32 publications

Which pooling method is better: Max, Avg, or Concat (Max, Avg)

Which pooling method is better: Max, Avg, or Concat (Max, Avg)

Electric Load Forecasting Based on Deep Ensemble Learning

Accurate Detection of Treeless Areas in Forests Using Deep Learning Models

Contact Info

Product

Resources

About