The transition module: a method for preventing overfitting in convolutional neural networks

Akbar, Shazia; Peikari, Mohammad; Salama, Sherine; Nofech‐Mozes, Sharon; Martel, Anne L.

doi:10.1080/21681163.2018.1427148

Cited by 12 publications

(9 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Dropout regularization is applied to mitigate the overfitting problems that traditional neural networks have. Recent studies used dropout regularization for enhancing DL testing results ( 43 , 51 , 52 ). When dropout regulation is applied at a hidden neural network layer, a random thinned neural network layer is produced from the original layer.…”

Section: Methodsmentioning

confidence: 99%

Deep Learning Framework for Freeway Speed Prediction in Adverse Weather

Shabarek

Chien

Hadri³

2020

Transportation Research Record

View full text Add to dashboard Cite

The introduction of deep learning (DL) models and data analysis may significantly elevate the performance of traffic speed prediction. Adverse weather causes mobility and safety concerns because of varying traffic speeds with poor visibility and road conditions. Most previous modeling approaches have not considered the heterogeneity of temporal and spatial data, such as traffic and weather conditions. This paper presents a framework, consisting of two DL models, to predict traffic speed under normal conditions and during adverse weather, considering prevailing traffic speed, wind speed, traffic volume, road capacity, wind bearing, precipitation intensity, and visibility. To ensure the accuracy of speed prediction, different DL models were assessed. The results indicated that the proposed one-dimensional convolutional neural network model outperformed others in relation to the least root mean square error and the least mean absolute error. Considering real-time weather data feeds on a 15-min basis, a tool was also developed for displaying predicted traffic speeds on New Jersey freeways. Application of the proposed framework models for predicting spatio-temporal hot-spot congestion caused by adverse weather is discussed.

show abstract

Section: Methodsmentioning

confidence: 99%

Deep Learning Framework for Freeway Speed Prediction in Adverse Weather

Shabarek

Chien

Hadri³

2020

Transportation Research Record

View full text Add to dashboard Cite

show abstract

“…In this study, we showed that the DeepSnap-DL approach enables high-throughput and high-quality prediction of the PR antagonist due to the automatic extraction of feature values from 3D-chemical structures adjusted as suitable input data into the DL, as well as avoiding overfitting through selective activation of molecular features with integration of multi-layered networks (Guo et al, 2017;Liang et al, 2017;Kong and Yu, 2018;Akbar et al, 2019). In addition, consist with recent reports (Chauhan et al, 2019;Cortés-Ciriano and Bender, 2019), this study indicated both the training data size and image redundancy are critical factors when determining prediction performance.…”

Section: Comparison Of the Predictionmentioning

confidence: 99%

DeepSnap-Deep Learning Approach Predicts Progesterone Receptor Antagonist Activity With High Performance

Matsuzaka

Uesawa

2020

Front. Bioeng. Biotechnol.

View full text Add to dashboard Cite

The progesterone receptor (PR) is important therapeutic target for many malignancies and endocrine disorders due to its role in controlling ovulation and pregnancy via the reproductive cycle. Therefore, the modulation of PR activity using its agonists and antagonists is receiving increasing interest as novel treatment strategy. However, clinical trials using the PR modulators have not yet been found conclusive evidences. Recently, increasing evidence from several fields shows that the classification of chemical compounds, including agonists and antagonists, can be done with recent improvements in deep learning (DL) using deep neural network. Therefore, we recently proposed a novel DL-based quantitative structure-activity relationship (QSAR) strategy using transfer learning to build prediction models for agonists and antagonists. By employing this novel approach, referred as DeepSnap-DL method, which uses images captured from 3-dimension (3D) chemical structure with multiple angles as input data into the DL classification, we constructed prediction models of the PR antagonists in this study. Here, the DeepSnap-DL method showed a high performance prediction of the PR antagonists by optimization of some parameters and image adjustment from 3Dstructures. Furthermore, comparison of the prediction models from this approach with conventional machine learnings (MLs) indicated the DeepSnap-DL method outperformed these MLs. Therefore, the models predicted by DeepSnap-DL would be powerful tool for not only QSAR field in predicting physiological and agonist/antagonist activities, toxicity, and molecular bindings; but also for identifying biological or pathological phenomena.

show abstract

“…The authors in [15] introduced a transition module that can capture filters at different scales, collapsing the filters via global average pooling. This ultimately reduces the size of the network from convolutional to fully connected layers.…”

Section: Related Workmentioning

confidence: 99%

Efficient, Ultra-facile Breast Cancer Histopathological Images Classification Approach Utilizing Deep Learning Optimizers

Kwadwo¹,

You²,

Tettey³

2020

IJCA

View full text Add to dashboard Cite

Conventional approaches to breast cancer diagnosis are associated with drawbacks that ultimately affect the quality of diagnosis and subsequent treatment, pushing for the need for automatic and precise classification of breast cancer tumors. The advent of deep learning methods has witnessed an increasing interest in their applications in many tasks. The specific case of using convolutional neural networks with transfer learning has witnessed tremendous successes in many classification tasks. Nonetheless, with transfer learning, the sheer number of parameters associated with deep networks coupled with the distance disparity between source data and target data leave networks prone to overfitting, particularly in the case of limited data. Also, negative transfer may occur in the situation where the source and target domains are not related. This work proposes a simple convolutional neural network model trained from scratch for discriminating benign and malignant breast cancer tumors in histopathological images. Four deep learning optimization algorithms are leveraged and explored to ascertain how optimizers aid in finding good sets of parameters that help minimize loss and increase overall classification accuracy. By adopting a polynomial learning rate decay scheduling and implementing several data augmentation techniques that regulate overfitting and improve the generalization ability of the proposed model, accuracy, sensitivity, specificity, and Area Under the Curve values of 89.92%, 94.02%, 86.42%, and 0.884 (88.4%), respectively are reported.

show abstract

The transition module: a method for preventing overfitting in convolutional neural networks

Cited by 12 publications

References 15 publications

Deep Learning Framework for Freeway Speed Prediction in Adverse Weather

Deep Learning Framework for Freeway Speed Prediction in Adverse Weather

DeepSnap-Deep Learning Approach Predicts Progesterone Receptor Antagonist Activity With High Performance

Efficient, Ultra-facile Breast Cancer Histopathological Images Classification Approach Utilizing Deep Learning Optimizers

Contact Info

Product

Resources

About