A Self-gated Activation Function SINSIG Based on the Sine Trigonometric for Neural Network Models

Douge, Khalid; Berrahou, Aissam; Alaoui, Youssef Talibi; Alaoui, Mohammed Talibi

doi:10.1007/978-3-030-70866-5_15

Cited by 1 publication

(2 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where σ(z) is the logistic sigmoid function [209]. While SinSig is similar to swish and mish, it outperformed them in experiments in [209] as the number of layers in a neural network increased. It was also shown that the SinSig converges faster.…”

Section: Sinsigmentioning

confidence: 99%

See 1 more Smart Citation

On Transformative Adaptive Activation Functions in Neural Networks for Gene Expression Inference

Kunc

Kléma²

2019

Preprint

View full text Add to dashboard Cite

Motivation: Gene expression profiling was made cheaper by the NIH LINCS program that profiles only ∼1, 000 selected landmark genes and uses them to reconstruct the whole profile. The D-GEX method employs neural networks to infer the whole profile. However, the original D-GEX can be further significantly improved. Results: We have analyzed the D-GEX method and determined that the inference can be improved using a logistic sigmoid activation function instead of the hyperbolic tangent. Moreover, we propose a novel transformative adaptive activation function that improves the gene expression inference even further and which generalizes several existing adaptive activation functions. Our improved neural network achieves average mean absolute error of 0.1340 which is a significant improvement over our reimplementation of the original D-GEX which achieves average mean absolute error 0.1637

show abstract

Section: Sinsigmentioning

confidence: 99%

“…It was also shown that the SinSig converges faster. The SinSig outperformed ReLU and mish on several deep architectures including ResNet 20 v2 [210], ResNet 110 v2 [210], SqueezeNet [211], and ShuffleNet [212] among others on the CIFAR-100 task [158] in experiments in [209].…”

Section: Sinsigmentioning

confidence: 99%

On Transformative Adaptive Activation Functions in Neural Networks for Gene Expression Inference

Kunc

Kléma²

2019

Preprint

View full text Add to dashboard Cite

show abstract

A Self-gated Activation Function SINSIG Based on the Sine Trigonometric for Neural Network Models

Cited by 1 publication

References 3 publications

On Transformative Adaptive Activation Functions in Neural Networks for Gene Expression Inference

On Transformative Adaptive Activation Functions in Neural Networks for Gene Expression Inference

Contact Info

Product

Resources

About