Smish: A Novel Activation Function for Deep Learning Methods

Wang, Xueliang; Ren, Hongxiang; A-chuan, Wang

doi:10.3390/electronics11040540

Cited by 42 publications

(19 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The recent research made by Jin et al (2022) [28] focuses on solving the Time-varying Sylvester equation using Zeroing Neural Network based on the Versatile Activation Function (VAF) variations which include exponential convergence characteristic and adjustable parameter. Another recent work proposed a function called Smish by Wang et al (2022) [29]. It was experimented on the CIFAR-10 dataset with the EfficientNetB3 network, on the MNIST dataset with the EfficientNetB5 network, and on the SVHN dataset with the EfficientnetB7 network.…”

Section: Related Workmentioning

confidence: 99%

The Impact of Irrationals on the Range of Arctan Activation Function for Deep Learning Models

Tümer-Sivri¹,

Akman²,

Berkol³

2023

Preprint

View full text Add to dashboard Cite

Deep learning has been applied in many areas that have had a significant impact on applications that improves real-life challenges. The success of deep learning in a wide range of areas is due in part to the use of activation functions, which are particularly effective at solving non-linear problems. Activation functions are a key focus for researchers in artificial intelligence who aim to improve the performance of neural networks. This article provides a comprehensive explanation and comparison of different activation functions, with a focus on the arc tangent and its variations specifically. The paper presents experimental results that show that variations of the arc tangent using irrational numbers such as pi, the golden ratio, and Euler’s number, as well as a self-arctan function, produce promising results. Since we experimented with promising activation functions on two different problems, and datasets, we reached a result that different irrationals work well for different problems. In other words, arctan ϕ gives the best results mostly for multiclass classification and arctan e gives the best results for time series prediction problems. The paper focuses on a multi-class classification problem applied to the Reuters Newswire dataset and a time-series prediction problem on Türkiye energy trade value to show the impacts of activation functions.

show abstract

Section: Related Workmentioning

confidence: 99%

The Impact of Irrationals on the Range of Arctan Activation Function for Deep Learning Models

Tümer-Sivri¹,

Akman²,

Berkol³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Its trainable parameter offers fine tuning for maximized output of propagation with a smooth gradient. This generates a faster, easier, and efficient generalizability of results [72].…”

Section: Algorithm 2: Artificial Neural Networkmentioning

confidence: 99%

Utilization of Random Forest Classifier and Artificial Neural Network for Predicting Factors Influencing the Perceived Usability of COVID-19 Contact Tracing “MorChana” in Thailand

Ong

Prasetyo

Yuduang

et al. 2022

IJERPH

View full text Add to dashboard Cite

With the constant mutation of COVID-19 variants, the need to reduce the spread should be explored. MorChana is a mobile application utilized in Thailand to help mitigate the spread of the virus. This study aimed to explore factors affecting the actual use (AU) of the application through the use of machine learning algorithms (MLA) such as Random Forest Classifier (RFC) and Artificial Neural Network (ANN). An integrated Protection Motivation Theory (PMT) and the Unified Theory of Acceptance and Use of Technology (UTAUT) were considered. Using convenience sampling, a total of 907 valid responses from those who answered the online survey were voluntarily gathered. With 93.00% and 98.12% accuracy from RFC and ANN, it was seen that hedonic motivation and facilitating conditions were seen to be factors affecting very high AU; while habit and understanding led to high AU. It was seen that when people understand the impact and causes of the COVID-19 pandemic’s aftermath, its severity, and also see a way to reduce it, it would lead to the actual usage of a system. The findings of this study could be used by developers, the government, and stakeholders to capitalize on using the health-related applications with the intention of increasing actual usage. The framework and methodology used presented a way to evaluate health-related technologies. Moreover, the developing trends of using MLA for evaluating human behavior-related studies were further justified in this study. It is suggested that MLA could be utilized to assess factors affecting human behavior and technology used worldwide.

show abstract

“…The Smish inherits the nonmonotonic properties of the Logish function. The Smish function is mostly suitable for machine learning networks which can be expressed in equation ( 2) [17],…”

Section: Egpnetmentioning

confidence: 99%

Coot optimization based Enhanced Global Pyramid Network for 3D hand pose estimation

Malavath

Devarakonda

2022

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

Due to its importance in various applications that need human-computer interaction (HCI), the field of 3D Hand Pose Estimation (HPE) has recently got a lot of attention. The use of technological developments, such as deep learning networks has accelerated the development of reliable 3D HPE systems. Therefore, in this paper, a 3D HPE based on Enhanced Global Pyramid Network (EGPNet) is proposed. Initially, feature extraction is done by backbone model of DetNet with improved EGPNet. The EGPNet is enhanced by the Smish activation function. After the feature extraction, the hand pose estimation is performed based on 3D pose correction network. Additionally, to enhance the estimation performance, Coot optimization algorithm is used to optimize the error between estimated and ground truth hand pose. The effectiveness of the proposed method is experimented on Bharatanatyam, yoga, Kathakali and sign language datasets with different networks in terms of AUC, Median EPE and Mean EPE. The coot optimization is also compared with existing optimization algorithms.

show abstract

Smish: A Novel Activation Function for Deep Learning Methods

Cited by 42 publications

References 30 publications

The Impact of Irrationals on the Range of Arctan Activation Function for Deep Learning Models

The Impact of Irrationals on the Range of Arctan Activation Function for Deep Learning Models

Utilization of Random Forest Classifier and Artificial Neural Network for Predicting Factors Influencing the Perceived Usability of COVID-19 Contact Tracing “MorChana” in Thailand

Coot optimization based Enhanced Global Pyramid Network for 3D hand pose estimation

Contact Info

Product

Resources

About