Exploring weight symmetry in deep neural networks

Hu, Shell Xu; Zagoruyko, Sergey; Komodakis, Nikos

doi:10.1016/j.cviu.2019.07.006

Cited by 13 publications

(7 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The symmetry of neural network weights was studied in [13]. It was shown that weights located at the same distance from the centre of the network can be equal or inversely proportional.…”

Section: Related Workmentioning

confidence: 99%

Influence of the Symmetry Neural Network Morphology on the Mine Detection Metric

Peleshchak,

Lytvyn,

Nazarkevych

et al. 2024

Symmetry

View full text Add to dashboard Cite

Presently, active detectors are widely used to detect mines, providing high accuracy. However, the principle of the operation of active detectors can lead to the explosion of hidden mines. The novelty of this work is the development of the morphology of a neural network for the classification of mines made of different materials (metallic, semi-metallic, plastic) with high accuracy (99.23%), based on a vector of input features with the following components: the value of the output voltage of the FLC-100 magnetic field sensor, which measures magnetic field anomalies in the vicinity of mines with an accuracy of 10−10–10−4 Tesla; six different soil types, depending on the humidity; and the height at which the magnetic field sensor is located above the mine. Due to the fact that mines, when made of different materials (metallic, semi-metallic, plastic), have different magnetic properties, the neural network method of mine classification, based on the sensor data regarding anomalies of the magnetic field in the vicinity of mines, allows the classification of mines made of different materials. The accuracy of mine classification was assessed with two-layer and three-layer neural networks on various metrics (confusion matrix, ROC curves, accuracy–loss curves), using ADAM, RMSprop, and SGD optimisers, and analyses and comparisons were then carried out. The impact of asymmetry in the neuron number and the types of activation functions in the first and second hidden layers on the values of the accuracy and loss metrics was studied. In particular, it was established that the asymmetry of the number of neurons in the first and second hidden layers relative to the plane of symmetry between the hidden layers has a significant effect on the accuracy of the model (decrease in accuracy by 25%), while the loss function, when the symmetry of the neurons number in the hidden layers is violated, increases to a maximum of 50%.

show abstract

“…The symmetry of neural network weights was studied in [13]. It was shown that weights located at the same distance from the centre of the network can be equal or inversely proportional.…”

Section: Related Workmentioning

confidence: 99%

Influence of the Symmetry Neural Network Morphology on the Mine Detection Metric

Peleshchak,

Lytvyn,

Nazarkevych

et al. 2024

Symmetry

View full text Add to dashboard Cite

show abstract

“…CNNs can capture richer and more complex characteristics by adjusting the network depth. However, the vanishing gradient [47] problem makes network training more difficult. The network can collect more fine-grained features by adjusting its width.…”

Section: Figure 3: Architecture Of the Efficientnet-b0mentioning

confidence: 99%

An Efficient and Robust Hand Gesture Recognition System of Sign Language Employing Finetuned Inception-V3 and Efficientnet-B0 Network

Hussain¹,

Amin²,

Fayaz³

et al. 2023

Computer Systems Science and Engineering

View full text Add to dashboard Cite

Hand Gesture Recognition (HGR) is a promising research area with an extensive range of applications, such as surgery, video game techniques, and sign language translation, where sign language is a complicated structured form of hand gestures. The fundamental building blocks of structured expressions in sign language are the arrangement of the fingers, the orientation of the hand, and the hand's position concerning the body. The importance of HGR has increased due to the increasing number of touchless applications and the rapid growth of the hearing-impaired population. Therefore, real-time HGR is one of the most effective interaction methods between computers and humans. Developing a user-free interface with good recognition performance should be the goal of real-time HGR systems. Nowadays, Convolutional Neural Network (CNN) shows great recognition rates for different image-level classification tasks. It is challenging to train deep CNN networks like VGG-16, VGG-19, Inception-v3, and Efficientnet-B0 from scratch because only some significant labeled image datasets are available for static hand gesture images. However, an efficient and robust hand gesture recognition system of sign language employing finetuned Inception-v3 and Efficientnet-Bo network is proposed to identify hand gestures using a comparative small HGR dataset. Experiments show that Inception-v3 achieved 90% accuracy and 0.93% precision, 0.91% recall, and 0.90% f1-score, respectively, while EfficientNet-B0 achieved 99% accuracy and 0.98%, 0.97%, 0.98%, precision, recall, and f1-score respectively.

show abstract

“…Therefore, the entire non-local spectral correlation prediction does not take up a lot of calculation and memory. After this operation, we also add weight symmetrization [36] to obtain a symmetric correlation matrix C s . The weight symmetrization can be briefly expressed by a linear operator [36].…”

Section: The Non-local Spatial-spectral Attention Modulementioning

confidence: 99%

“…After this operation, we also add weight symmetrization [36] to obtain a symmetric correlation matrix C s . The weight symmetrization can be briefly expressed by a linear operator [36]. Subsequently, the feature map x is subjected to 1 × 1 convolution processing and then multiplication with C s .…”

Section: The Non-local Spatial-spectral Attention Modulementioning

confidence: 99%

Hyperspectral Snapshot Compressive Imaging with Non-Local Spatial-Spectral Residual Network

et al. 2021

View full text Add to dashboard Cite

Snapshot Compressive Imaging is an emerging technology that is based on compressive sensing theory to achieve high-efficiency hyperspectral data acquisition. The core problem of this technology is how to reconstruct 3D hyperspectral data from the 2D snapshot measurement in a fast and high-quality manner. In this paper, we propose a novel deep network, which consists of the symmetric residual module and the non-local spatial-spectral attention module, to learn the reconstruction mapping in a data-driven way. The symmetric residual module uses symmetric residual connections to improve the potential of interaction between convolution operations and further promotes the fusion of local features. The non-local spatial-spectral attention module is designed to capture the non-local spatial-spectral correlation in the hyperspectral image. Specifically, this module calculates the channel attention matrix to capture the global correlations between all of the spectral channels, and it fuses the channel attention attained feature maps and the spatial attention weighted features as the module output, thus both of the spatial-spectral correlations of hyperspectral images can be fully utilized for reconstruction. In addition, a compound loss, including the reconstruction loss, the measurement loss, and the cosine loss, is designed to guide the end-to-end network learning. We experimentally evaluate the proposed method on simulation and real datasets. The experimental results show that the proposed network outperforms the competing methods in terms of the reconstruction quality and running time.

show abstract

Exploring weight symmetry in deep neural networks

Cited by 13 publications

References 23 publications

Influence of the Symmetry Neural Network Morphology on the Mine Detection Metric

Influence of the Symmetry Neural Network Morphology on the Mine Detection Metric

An Efficient and Robust Hand Gesture Recognition System of Sign Language Employing Finetuned Inception-V3 and Efficientnet-B0 Network

Hyperspectral Snapshot Compressive Imaging with Non-Local Spatial-Spectral Residual Network

Contact Info

Product

Resources

About