Weight initialization based‐rectified linear unit activation function to improve the performance of a convolutional neural network model

Olimov, Bekhzod; Sanjar, Karshiev; Jang, Eungyeong; Din, Sadia; Paul, Anand; Kim, Jeonghong

doi:10.1002/cpe.6143

Cited by 24 publications

(16 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…After stage 1, the extracted features, that is, the landmark points ( ) per frame are flattened, concatenated and stored in a file to check and remove any null entries from the data. Data cleaning is important since it prevents failed detection of features 56 – 58 , which occurs when a blurred image is sent to the detector and leads to a null entry into the dataset. Thus, when training occurs with this noisy data, the prediction accuracy is reduced and bias may occur.…”

Section: Proposed Methodologymentioning

confidence: 99%

An integrated mediapipe-optimized GRU model for Indian sign language recognition

Subramanian

Olimov

Naik

et al. 2022

Sci Rep

Self Cite

View full text Add to dashboard Cite

Sign language recognition is challenged by problems, such as accurate tracking of hand gestures, occlusion of hands, and high computational cost. Recently, it has benefited from advancements in deep learning techniques. However, these larger complex approaches cannot manage long-term sequential data and they are characterized by poor information processing and learning efficiency in capturing useful information. To overcome these challenges, we propose an integrated MediaPipe-optimized gated recurrent unit (MOPGRU) model for Indian sign language recognition. Specifically, we improved the update gate of the standard GRU cell by multiplying it by the reset gate to discard the redundant information from the past in one screening. By obtaining feedback from the resultant of the reset gate, additional attention is shown to the present input. Additionally, we replace the hyperbolic tangent activation in standard GRUs with exponential linear unit activation and SoftMax with Softsign activation in the output layer of the GRU cell. Thus, our proposed MOPGRU model achieved better prediction accuracy, high learning efficiency, information processing capability, and faster convergence than other sequential models.

show abstract

Section: Proposed Methodologymentioning

confidence: 99%

An integrated mediapipe-optimized GRU model for Indian sign language recognition

Subramanian

Olimov

Naik

et al. 2022

Sci Rep

Self Cite

View full text Add to dashboard Cite

show abstract

“…The batch normalization reduces the internal co-variant shift and also regularizes the model. A rectified linear unit (ReLU) [ 24 , 25 , 26 , 27 ] activation function is applied. Two advantages accompany the ReLU activation function: (1) It realizes the sparse representation of the network.…”

Section: Methodsmentioning

confidence: 99%

FECC-Net: A Novel Feature Enhancement and Context Capture Network Based on Brain MRI Images for Lesion Segmentation

Huang

Zhang

Song³

et al. 2022

Brain Sciences

View full text Add to dashboard Cite

In recent years, the increasing incidence of morbidity of brain stroke has made fast and accurate segmentation of lesion areas from brain MRI images important. With the development of deep learning, segmentation methods based on the computer have become a solution to assist clinicians in early diagnosis and treatment planning. Nevertheless, the variety of lesion sizes in brain MRI images and the roughness of the boundary of the lesion pose challenges to the accuracy of the segmentation algorithm. Current mainstream medical segmentation models are not able to solve these challenges due to their insufficient use of image features and context information. This paper proposes a novel feature enhancement and context capture network (FECC-Net), which is mainly composed of an atrous spatial pyramid pooling (ASPP) module and an enhanced encoder. In particular, the ASPP model uses parallel convolution operations with different sampling rates to enrich multi-scale features and fully capture image context information in order to process lesions of different sizes. The enhanced encoder obtains deep semantic features and shallow boundary features in the feature extraction process to achieve image feature enhancement, which is helpful for restoration of the lesion boundaries. We divide the pathological image into three levels according to the number of pixels in the real mask area and evaluate FECC-Net on an open dataset called Anatomical Tracings of Lesions After Stroke (ATLAS). The experimental results show that our FECC-Net outperforms mainstream methods, such as DoubleU-Net and TransUNet. Especially in small target tasks, FECC-Net is 4.09% ahead of DoubleU-Net on the main indicator DSC. Therefore, FECC-Net is encouraging and can be relied upon for brain MRI image applications.

show abstract

“…5 ). The ReLU (Rectified Linear Unit) activation function 39 is assigned to neurons in all convolution layers and fully-connected layer, while the Softmax activation function is assigned to neurons in the last layer to output the classification results. The filter with the size of 3 × 3 is used to expand the number of channels to extract expressive and complex features, and the output data has the same size as the input data through the numeral zero padding.…”

Section: Network Architecture and Applied Strategiesmentioning

confidence: 99%

Pet dog facial expression recognition based on convolutional neural network and improved whale optimization algorithm

Mao

Liu

2023

Sci Rep

View full text Add to dashboard Cite

Pet dogs are our good friends. Realizing the dog’s emotions through the dog's facial expressions is beneficial to the harmonious coexistence between human beings and pet dogs. This paper describes a study on dog facial expression recognition using convolutional neural network (CNN), which is a representative algorithm model of deep learning. Parameter settings have a profound impact on the performance of a CNN model, improper parameter setting will make the model exposes several shortcomings, such as slow learning speed, easy to fall into local optimal solution, etc. In response to these shortcomings and improve the accuracy of recognition, a novel CNN model based on the improved whale optimization algorithm (IWOA) called IWOA–CNN is applied to complete this recognition task. Unlike human face recognition, a dedicated face detector in Dlib toolkit is utilized to recognize the facial region, and the captured facial images are augmented to build an expression dataset. The random dropout layer and L2 regularization are introduced into the network to reduce the number of transmission parameters of network and avoid over fitting. The IWOA optimizes the keep probability of the dropout layer, the parameter λ of L2 regularization and the dynamic learning rate of gradient descent optimizer. Carry out a comparative experiment of IWOA–CNN, Support Vector Machine, LeNet-5 and other classifiers for facial expression recognition, its results demonstrate that the IWOA–CNN has better recognition effect in facial expression recognition and also explain the efficiency of the swarm intelligence algorithm in dealing with model parameter optimization.

show abstract

Weight initialization based‐rectified linear unit activation function to improve the performance of a convolutional neural network model

Cited by 24 publications

References 17 publications

An integrated mediapipe-optimized GRU model for Indian sign language recognition

An integrated mediapipe-optimized GRU model for Indian sign language recognition

FECC-Net: A Novel Feature Enhancement and Context Capture Network Based on Brain MRI Images for Lesion Segmentation

Pet dog facial expression recognition based on convolutional neural network and improved whale optimization algorithm

Contact Info

Product

Resources

About