Towards Fast and Understandable Computations: Which “And”- and “Or”-Operations Can Be Represented by the Fastest (i.e., 1-Layer) Neural Networks? Which Activations Functions Allow Such Representations?

Alvarez, Kevin; Urenda, Julio C.; Csiszár, Orsolya; Csiszár, Gábor; Dombi, József; Eigner, György; Kreinovich, Vladik

doi:10.12700/aph.18.2.2021.2.2

Cited by 7 publications

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Combining extreme learning machines with the continuous logical background can be a very promising direction towards a more interpretable, transparent, and safe machine learning. Supplemental research is also in progress aiming to investe which "And"-and "Or"-operations can be represented by the fastest (i.e., 1-Layer) neural networks, and which activations functions allow such representations [1].…”

Section: Discussionmentioning

confidence: 99%

“…Note that the general operator for ν = 1 is conjunctive, for ν = 0 it is disjunctive and for ν = ν * = f −1 1 2 it is self-dual. As a benefit of using this general operator, a conjunction, a disjunction and an aggregative operator differ only in one parameter of the general operator in Equation (1). Additionally, the parameter ν has the semantic meaning of the level of expectation: maximal for the conjunction, neutral for the aggregation, and minimal for the disjunction.…”

Section: Nilpotent Logical Systemsmentioning

confidence: 99%

“…However, in high-risk environments, such as the health care or the insurance sector, it is important to be able to explain why a decision was made. In this case, we need to have some reasonable explanations behind our decisions to be more convincing and also to avoid lawsuits claiming race-based, gender-based, or age-based bias [1]. Understandability means that we are able to describe the computations by using words from natural human language.…”

Section: Introductionmentioning

confidence: 99%

“…The basic idea of continuous logic is the replacement of the space of truth values {T, F } by a compact interval such as [0,1]. This means that the inputs and the outputs of the extended logical gates are real numbers of the unit interval, representing truth values of inequalities.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Squashing activation functions in benchmark tests: towards eXplainable Artificial Intelligence using continuous-valued logic

Zeltner¹,

Schmid²,

Csiszár³

et al. 2020

Preprint

View full text Add to dashboard Cite

Over the past few years, deep neural networks have shown excellent results in multiple tasks, however, there is still an increasing need to address the problem of interpretability to improve model transparency, performance, and safety. Achieving eXplainable Artificial Intelligence (XAI) by combining neural networks with continuous logic and multi-criteria decision-making tools is one of the most promising ways to approach this problem: by this combination, the black-box nature of neural models can be reduced. The continuous logic-based neural model uses so-called Squashing activation functions, a parametric family of functions that satisfy natural invariance requirements and contain rectified linear units as a particular case. This work demonstrates the first benchmark tests that measure the performance of Squashing functions in neural networks. Three experiments were carried out to examine their usability and a comparison with the most popular activation functions was made for five different network types. The performance was determined by measuring the accuracy, loss and time per epoch. These experiments and the conducted benchmarks have proven that the use of Squashing functions is possible and similar in performance to conventional activation functions. Moreover, a further experiment was conducted by implementing nilpotent logical gates to demonstrate how simple classification tasks can be solved successfully and with high performance. The results indicate that due to the embedded nilpotent logical operators and the differentiability of the Squashing function, it is possible to solve classification problems, where other commonly used activation functions fail.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Nilpotent Logical Systemsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Squashing activation functions in benchmark tests: towards eXplainable Artificial Intelligence using continuous-valued logic

Zeltner¹,

Schmid²,

Csiszár³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…More specifically, "and" neurons are implemented through t-norms while "or" neurons are implemented using t-conorms (s-norms). In [28], the authors analyzed which versions of fuzzy techniques provide the best approximation to neural data processingand, vice versa, which activation function makes the results of neural processing best describable in fuzzy terms. Interestingly, they concluded that the best activation function in this sense is exactly the ReLU function.…”

Section: Neural Network Based On Nilpotent Fuzzy Logic and Mcdm Toolsmentioning

confidence: 99%

Uninorm-like parametric activation functions for human-understandable neural models

Csiszár¹,

Sára²,

Dénes-Fazakas³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

We present a deep learning model for finding human-understandable connections between input features. Our approach uses a parameterized, differentiable activation function, based on the theoretical background of nilpotent fuzzy logic and multi-criteria decision-making (MCDM). The learnable parameter has a semantic meaning indicating the level of compensation between input features. The neural network determines the parameters using gradient descent to find human-understandable relationships between input features. We demonstrate the utility and effectiveness of the model by successfully applying it to classification problems from the UCI Machine Learning Repository.

show abstract

Data Processing: Neural and Fuzzy Techniques

Urenda

Kreinovich²

2022

Studies in Big Data

View full text Add to dashboard Cite

Towards Fast and Understandable Computations: Which “And”- and “Or”-Operations Can Be Represented by the Fastest (i.e., 1-Layer) Neural Networks? Which Activations Functions Allow Such Representations?

Cited by 7 publications

References 0 publications

Squashing activation functions in benchmark tests: towards eXplainable Artificial Intelligence using continuous-valued logic

Squashing activation functions in benchmark tests: towards eXplainable Artificial Intelligence using continuous-valued logic

Uninorm-like parametric activation functions for human-understandable neural models

Data Processing: Neural and Fuzzy Techniques

Contact Info

Product

Resources

About