A novel optimized parametric hyperbolic tangent swish activation function for 1D-CNN: application of sensor-based human activity recognition and anomaly detection

Ankalaki, Shilpa; Thippeswamy, M. N.

doi:10.1007/s11042-023-15766-3

Cited by 7 publications

(2 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Selecting the appropriate activation function is a challenging task. The authors of [35] studied the impact of activation function on the performance of CNN and proposed a more non-linear activation function called OP-Tanish. FC layers process high-level features from the convolutional layers and make final predictions.…”

Section: Overview Of Cnnmentioning

confidence: 99%

Optimized Convolutional Neural Network Using Hierarchical Particle Swarm Optimization for Sensor Based Human Activity Recognition

Ankalaki,

Thippeswamy

2024

SN COMPUT. SCI.

Self Cite

View full text Add to dashboard Cite

Hyperparameter optimization poses a significant challenge when developing deep neural networks. Building a convolutional neural network (CNN) for implementation can be an arduous and time-intensive task. This work proposed an approach to optimize the hyperparameters of one dimensional (1D-CNN) to improve the accuracy of human activity recognition (HAR). The framework includes a parametric depiction of 1D-CNNs along with an optimization process for hyperparameters aimed at maximizing the model's performance. This work designed the method called OPTConvNet for hyperparameter optimization of 1D-CNN using Hierarchical Particle Swarm Optimization (H-PSO). The H-PSO algorithm is designed to optimize the architectural, layer and training parameters of 1D-CNN. The H-PSO optimizes the architecture of the 1D-CNN at initial level. Layer and training hyperparameters will be optimized at the next level. The proposed approach employs an exponential-like inertia weight to fine-tune the balance between exploration and exploitation of particles to prevent premature convergence to a local optimum solution in the PSO algorithm. The H-PSO- CNN is evaluated on publicly available sensor- human activity recognition (S-HAR) datasets namely, UCI-HAR, Daphnet Gait, Opportunity and PAMPA2 datasets.

show abstract

Section: Overview Of Cnnmentioning

confidence: 99%

Optimized Convolutional Neural Network Using Hierarchical Particle Swarm Optimization for Sensor Based Human Activity Recognition

Ankalaki,

Thippeswamy

2024

SN COMPUT. SCI.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The NMAF [60] performs better than ELU, but the structure is more complex than ELU. The OP-Tanish [61] gets high accuracy but a long calculation time. Hwang et al [62] proposed a universal activation function, which exhibits the properties of activation functions such as ReLU, Sigmoid, and Swish by adjusting three hyperparameters.…”

Section: Introductionmentioning

confidence: 99%

PolyLU: A Simple and Robust Polynomial-Based Linear Unit Activation Function for Deep Learning

Feng,

Yang

2023

IEEE Access

View full text Add to dashboard Cite

The activation function has a critical influence on whether a convolutional neural network in deep learning can converge or not; a proper activation function not only makes the convolutional neural network converge faster but also can reduce the complexity of convolutional neural network architecture and gets the same or better performance. Many activation functions have been proposed; however, various activation functions have advantages, defects, and applicable network architectures. A new activation function called Polynomial Linear Unit (PolyLU) is proposed in this paper to improve some of the shortcomings of the existing activation functions. The PolyLU meets the following basic properties: continuously differentiable, approximate identity near the origin, unbounded for positive inputs, bounded for negative inputs, smooth, monotonic, and zero-centered. There is a polynomial term for the negative inputs and no exponential terms in the PolyLU that reduces the computational complexity of the network. Compared to those common activation functions like Sigmoid, Tanh, ReLU, LeakyReLU, ELU, Mish, and Swish, the experiments show that the PolyLU has improved some network complexity and has better accuracy over MNIST, Kaggle Cats and Dogs, CIFAR-10 and CIFAR-100 datasets. Test by the CIFAR-100 dataset with batch normalization, PolyLU improves by 0.

show abstract

Reparameterized dilated architecture: A wider field of view for pedestrian detection

Gong,

Huang,

Chen

et al. 2024

Appl Intell

View full text Add to dashboard Cite

A novel optimized parametric hyperbolic tangent swish activation function for 1D-CNN: application of sensor-based human activity recognition and anomaly detection

Cited by 7 publications

References 41 publications

Optimized Convolutional Neural Network Using Hierarchical Particle Swarm Optimization for Sensor Based Human Activity Recognition

Optimized Convolutional Neural Network Using Hierarchical Particle Swarm Optimization for Sensor Based Human Activity Recognition

PolyLU: A Simple and Robust Polynomial-Based Linear Unit Activation Function for Deep Learning

Reparameterized dilated architecture: A wider field of view for pedestrian detection

Contact Info

Product

Resources

About