A comparison of deep networks with ReLU activation function and linear spline-type methods

Eckle, Konstantin; Schmidt-Hieber, Johannes

doi:10.1016/j.neunet.2018.11.005

Cited by 303 publications

(141 citation statements)

References 13 publications

(24 reference statements)

Supporting

Mentioning

120

Contrasting

Order By: Relevance

“…One main difference between our model and RED is the improvement in the activation functions, which was changed from a Rectified Linear Unit (ReLU) to a Leaky Rectified Linear Unit (LeakyReLU) for all convolutional and deconvolutional layers except the final deconvolutional layer, which use Clipped Rectified Linear Unit as the activation function to limit values in the range [0, 1]. Previous studies have shown that different activation functions have an impact on the final performance of a CNN [ 67 , 68 ]. Hence, the improvement in the activation functions contributed to the better image restoration by the SR network.…”

Section: Methodsmentioning

confidence: 99%

Identifying Facemask-Wearing Condition Using Image Super-Resolution with Classification Network to Prevent COVID-19

Qin

2020

Sensors

197

View full text Add to dashboard Cite

The rapid worldwide spread of Coronavirus Disease 2019 (COVID-19) has resulted in a global pandemic. Correct facemask wearing is valuable for infectious disease control, but the effectiveness of facemasks has been diminished, mostly due to improper wearing. However, there have not been any published reports on the automatic identification of facemask-wearing conditions. In this study, we develop a new facemask-wearing condition identification method by combining image super-resolution and classification networks (SRCNet), which quantifies a three-category classification problem based on unconstrained 2D facial images. The proposed algorithm contains four main steps: Image pre-processing, facial detection and cropping, image super-resolution, and facemask-wearing condition identification. Our method was trained and evaluated on the public dataset Medical Masks Dataset containing 3835 images with 671 images of no facemask-wearing, 134 images of incorrect facemask-wearing, and 3030 images of correct facemask-wearing. Finally, the proposed SRCNet achieved 98.70% accuracy and outperformed traditional end-to-end image classification methods using deep learning without image super-resolution by over 1.5% in kappa. Our findings indicate that the proposed SRCNet can achieve high-accuracy identification of facemask-wearing conditions, thus having potential applications in epidemic prevention involving COVID-19.

show abstract

Section: Methodsmentioning

confidence: 99%

Identifying Facemask-Wearing Condition Using Image Super-Resolution with Classification Network to Prevent COVID-19

Qin

2020

Sensors

197

View full text Add to dashboard Cite

show abstract

“…One of the most promising new areas in artificial intelligence (AI) and machine learning for building predictive models are the so‐called deep learning technologies, [ 17 ] which are extensions of the traditional neural networks architectures, using more hidden layers and a larger variety of activation functions, [ 18 ] that is, functions that map the input to the output response of each neuron. Convolutional neural networks (CNNs) have shown state‐of‐the‐art performance for image classification, segmentation, and object detection and tracking.…”

Section: Introductionmentioning

confidence: 99%

Development of Deep Learning Models for Predicting the Effects of Exposure to Engineered Nanomaterials on Daphnia magna

Karatzas

Melagraki

Ellis

et al. 2020

Small

View full text Add to dashboard Cite

This study presents the results of applying deep learning methodologies within the ecotoxicology field, with the objective of training predictive models that can support hazard assessment and eventually the design of safer engineered nanomaterials (ENMs). A workflow applying two different deep learning architectures on microscopic images of Daphnia magna is proposed that can automatically detect possible malformations, such as effects on the length of the tail, and the overall size, and uncommon lipid concentrations and lipid deposit shapes, which are due to direct or parental exposure to ENMs. Next, classification models assign specific objects (heart, abdomen/claw) to classes that depend on lipid densities and compare the results with controls. The models are statistically validated in terms of their prediction accuracy on external D. magna images and illustrate that deep learning technologies can be useful in the nanoinformatics field, because they can automate time‐consuming manual procedures, accelerate the investigation of adverse effects of ENMs, and facilitate the process of designing safer nanostructures. It may even be possible in the future to predict impacts on subsequent generations from images of parental exposure, reducing the time and cost involved in long‐term reproductive toxicity assays over multiple generations.

show abstract

“…The values of the mask were then determined to minimize the loss through iterative learning. In this study, as an activation function, the rectified linear unit (ReLU) function [43][44][45][46] rather than the Sigmoid function is used as shown in Figure 7. That is because a vanishing gradient (in which a gradient converges to zero) occurs if the Sigmoid function is used [47][48][49].…”

Section: Detection Of Target Regionmentioning

confidence: 99%

Robust Blocking of Human Faces with Personal Information Using Artificial Deep Neural Computing

Jang

Lee

2020

Sustainability

View full text Add to dashboard Cite

High-speed wired and wireless Internet are one of the useful ways to acquire various types of media data easily. In this circumstance, people also can easily get media data including objects with exposed personal information through the Internet. Exposure of personal information emerges as a social issue. This paper proposes an effective blocking technique that makes it possible to robustly detect target objects with exposed personal information from various types of input images with the use of deep neural computing and to effectively block the detected objects' regions. The proposed technique first utilizes the neural computing-based learning algorithm to robustly detect the target object including personal information from an image. It next generates a grid-type mosaic and lets the mosaic overlap the target object region detected in the previous step so as to effectively block the object region that includes personal information. Experimental results reveal that the proposed algorithm robustly detects the target object region with exposed personal information from a variety of input images and effectively blocks the detected region through grid-type mosaic processing. The object blocking technique proposed in this paper is expected to be applied to various application fields such as image security, sustainable anticipatory computing, object tracking, and target blocking.

show abstract

A comparison of deep networks with ReLU activation function and linear spline-type methods

Cited by 303 publications

References 13 publications

Identifying Facemask-Wearing Condition Using Image Super-Resolution with Classification Network to Prevent COVID-19

Identifying Facemask-Wearing Condition Using Image Super-Resolution with Classification Network to Prevent COVID-19

Development of Deep Learning Models for Predicting the Effects of Exposure to Engineered Nanomaterials on Daphnia magna

Robust Blocking of Human Faces with Personal Information Using Artificial Deep Neural Computing

Contact Info

Product

Resources

About