LipARELU: ARELU Networks aided by Lipschitz Acceleration

Mediratta, Ishita; Saha, Snehanshu; Mathur, S.S.

doi:10.1109/ijcnn52387.2021.9533853

Search citation statements

Order By: Relevance

Paper Sections

Select...

Approximate Relu (Apprelu)1

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2022

2024

Publication Types

Select...

Book2

Preprint1

Article1

Relationship

Self Cite0

Independent4

Authors

Journals

Cited by 4 publications

(1 citation statement)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The approximate ReLU (AppReLU) 19 [286,287] is the RePU with aditional scaling parameter; it is defined as…”

Section: Approximate Relu (Apprelu)mentioning

confidence: 99%

On Transformative Adaptive Activation Functions in Neural Networks for Gene Expression Inference

Kunc

Kléma²

2019

Preprint

View full text Add to dashboard Cite

Motivation: Gene expression profiling was made cheaper by the NIH LINCS program that profiles only ∼1, 000 selected landmark genes and uses them to reconstruct the whole profile. The D-GEX method employs neural networks to infer the whole profile. However, the original D-GEX can be further significantly improved. Results: We have analyzed the D-GEX method and determined that the inference can be improved using a logistic sigmoid activation function instead of the hyperbolic tangent. Moreover, we propose a novel transformative adaptive activation function that improves the gene expression inference even further and which generalizes several existing adaptive activation functions. Our improved neural network achieves average mean absolute error of 0.1340 which is a significant improvement over our reimplementation of the original D-GEX which achieves average mean absolute error 0.1637

show abstract

“…The approximate ReLU (AppReLU) 19 [286,287] is the RePU with aditional scaling parameter; it is defined as…”

Section: Approximate Relu (Apprelu)mentioning

confidence: 99%