Application of constrained learning in making deep networks more transparent, regularized, and biologically plausible

Ghiassirad, Hossein Ali; Shoorehdeli, Mahdi Aliyari; Farivar, Faezeh

doi:10.1016/j.engappai.2019.06.022

Cited by 6 publications

(3 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Both CNNs and M-CNNs should update their weights after forward passes of data through the neural network. The weights can be adjusted to reduce the difference between the actual and predicted classification results for subsequent forward passes [44]. Unlike CNNs, M-CNNs have to additionally calculate the difference between G + and G − to obtain the current weights (W 1 − W n ).…”

Section: Overall Learning Process Of M-cnns and Cnnsmentioning

confidence: 99%

Memristor-Based CNNs for Detecting Stress Using Brain Imaging Signals

Bak,

Park,

Lee

et al. 2024

IEEE Trans. Emerg. Top. Comput. Intell.

View full text Add to dashboard Cite

Typical convolutional neural networks (CNNs) are widely used to recognize a user's stress state using the functional near-infrared spectroscopy (fNIRS), which is the latest brain imaging technology. fNIRS signals are usually fed into CNN models in the form of high-dimensional image data. However, this approach is not easy to achieve high classification accuracy because of physiological noises in brain signals. It is also likely to overlook the process of evaluating the reliability of calculated classification accuracy. To solve these problems, we proposed a memristor-based CNN (M-CNNs) This model's weight update process involves using stochastic gradient descent with momentum (SGDM), where the normalized conductances of memristors are used as weight substitutes. These conductances are then adjusted to classify stress states. We calculated the classification accuracies between the control and stress groups by using the M-CNNs, and then compared them with those of the CNNs. We used DenseNet, the most recent CNN model, to simulate accuracy under the same conditions. To ensure a fair comparison, we divided the DenseNet into the memristorbased DenseNet (M-DenseNet) and the conventional DenseNet (C-DenseNet). As a result, we discovered that the accuracy of M-CNNs (93.33%) exceeded that of CNNs (87.50%), and is reliable by precision, recall, and F-Score calculated from a confusion matrix. Likewise, M-DenseNet (92.38%) has higher accuracy than C-DenseNet (90.00%), but shows lower accuracy than M-CNNs. Moreover, we observed the reproducibility of M-CNN/DenseNet in various datasets. Therefore, our study suggests a promising application of CNN by combining conductances of memristor for classifying stress states.

show abstract

Section: Overall Learning Process Of M-cnns and Cnnsmentioning

confidence: 99%

Memristor-Based CNNs for Detecting Stress Using Brain Imaging Signals

Bak,

Park,

Lee

et al. 2024

IEEE Trans. Emerg. Top. Comput. Intell.

View full text Add to dashboard Cite

show abstract

“…This DNNs classifier has four hidden layers. There is no specific design rule for DNNs, and this paper uses the integer power of two as the number of neural nodes for each hidden Each hidden layer consists of one fully connected layer, one relu activation layer [42] [43], and one dropout layer. The dropout ratios for Setting 1 are 20%, 30%, 40%, and 50%.…”

Section: Architectures and Mathematical Expressions Of The Dnns Classifiermentioning

confidence: 99%

Non-intrusive classification of gas-liquid flow regimes in an S-shaped pipeline riser using a Doppler ultrasonic sensor and deep neural networks

Nnabuife

Kuang

Whidborne

et al. 2021

Chemical Engineering Journal

View full text Add to dashboard Cite

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

show abstract

“…Stochastic multiplicative gradient descent 12 Kernel Trick 13 Element-wise 14 Residual block 15 Epoch…”

mentioning

confidence: 99%

To Analysis and Compare 21 Weight Constraints in Stochastic Gradient Descent Algorithm Using Kernel Method

Ghiassirad

Shoorehdeli²,

Farivar³

2022

Journal of Iranian Association of Electrical and Electronics En

Self Cite

View full text Add to dashboard Cite

Application of constrained learning in making deep networks more transparent, regularized, and biologically plausible

Cited by 6 publications

References 17 publications

Memristor-Based CNNs for Detecting Stress Using Brain Imaging Signals

Memristor-Based CNNs for Detecting Stress Using Brain Imaging Signals

Non-intrusive classification of gas-liquid flow regimes in an S-shaped pipeline riser using a Doppler ultrasonic sensor and deep neural networks

To Analysis and Compare 21 Weight Constraints in Stochastic Gradient Descent Algorithm Using Kernel Method

Contact Info

Product

Resources

About