AdderNet: Do We Really Need Multiplications in Deep Learning?

Chen, Hanting; Wang, Yunhe; Xu, Chunjing; Shi, Boxin; Xu, Chao; Tian, Qi; Xu, Chang

doi:10.1109/cvpr42600.2020.00154

Cited by 154 publications

(112 citation statements)

References 28 publications

Supporting

Mentioning

108

Contrasting

Order By: Relevance

“…Over the years, these basic model families produced several versions [74] and they have been extensively used by other researchers to develop modified and hybrid models [76] . Recent studies attempted to improve performance of the base models by proposing new layers and filters such as Sparse Shift Filter [77] , Asymmetric Convolution Block [78] , Adder Networks [79] , Virtual Pooling [80] , Discrete Wavelet Transform [81] , and HetConv [82] , etc. Some recent substantial models have been developed based on the base models, such as Res2Net [83] and Wide ResNet [84] using the ResNet model; while Log Dense Net [85] and Sparse Net [86] using the DenseNet model.…”

Section: Model Developmentmentioning

confidence: 99%

COVID-19 Detection Through Transfer Learning Using Multimodal Imaging Data

et al. 2020

View full text Add to dashboard Cite

Detecting COVID-19 early may help in devising an appropriate treatment plan and disease containment decisions. In this study, we demonstrate how transfer learning from deep learning models can be used to perform COVID-19 detection using images from three most commonly used medical imaging modes X-Ray, Ultrasound, and CT scan. The aim is to provide over-stressed medical professionals a second pair of eyes through intelligent deep learning image classification models. We identify a suitable Convolutional Neural Network (CNN) model through initial comparative study of several popular CNN models. We then optimize the selected VGG19 model for the image modalities to show how the models can be used for the highly scarce and challenging COVID-19 datasets. We highlight the challenges (including dataset size and quality) in utilizing current publicly available COVID-19 datasets for developing useful deep learning models and how it adversely impacts the trainability of complex models. We also propose an image pre-processing stage to create a trustworthy image dataset for developing and testing the deep learning models. The new approach is aimed to reduce unwanted noise from the images so that deep learning models can focus on detecting diseases with specific features from them. Our results indicate that Ultrasound images provide superior detection accuracy compared to X-Ray and CT scans. The experimental results highlight that with limited data, most of the deeper networks struggle to train well and provides less consistency over the three imaging modes we are using. The selected VGG19 model, which is then extensively tuned with appropriate parameters, performs in considerable levels of COVID-19 detection against pneumonia or normal for all three lung image modes with the precision of up to 86% for X-Ray, 100% for Ultrasound and 84% for CT scans.

show abstract

Section: Model Developmentmentioning

confidence: 99%

COVID-19 Detection Through Transfer Learning Using Multimodal Imaging Data

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Since min / max operations are not differentiable, PConv, LMorph and SMorph layers with continuous structure were proposed [65], [66]. Chen et al [67] modify only convolutional layers to use the L1-norm instead of covariation. It dramatically simplifies convolutional layers; however, other layers still use multiplication.…”

Section: B Alternative Neurons/layersmentioning

confidence: 99%

Bipolar Morphological Neural Networks: Gate-Efficient Architecture for Computer Vision

Limonova¹,

Alfonso²,

Nikolaev³

et al. 2021

IEEE Access

View full text Add to dashboard Cite

The priority of building hardware-oriented neural network models is growing steadily. The target goals for their development are the performance and energy efficiency of promising hardwaresoftware solutions. Simultaneously, for different classes of computing architectures of the computer, the optimal neural network models will differ. The most interesting from a practical point of view are applicationspecific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) and central processing units (CPUs). We have recently proposed a bipolar morphological network as a hardware-oriented model for these computer types, the computationally intensive parts of which use only maximum and addition. In this work, we present for the first time a theoretical assessment of the expressive power of a neural network consisting of BM neurons and show that it corresponds to the expressive power of the classical multilayer perceptron. In addition, we summarize the current results on the use of the bipolar morphological model in typical tasks of technical vision: image classification and semantic segmentation. We consider simple LeNet-5-like neural networks, as well as deeper ResNet and UNet architectures. We show that BM networks demonstrate accuracy that allows their practical use, with significantly higher performance in terms of a transistor budget for two (ASIC, FPGA) of the three architectures under consideration. The source code of the model and ResNet experiments are available at https://github.com/SmartEngines/bipolar-morphological-resnet.

show abstract

“…audio features are improved continuously to achieve adequate representation and be suitable to feature extraction in deep learning. (2). a more suitable classifier is proposed for audio classification.…”

Section: ⅵ Conclusionmentioning

confidence: 99%

“…Previous research of audio recognition was based on template matching [2], feature parameter matching [3] and hidden Markov methods [4]. The above methods achieve good performance in the early speech recognition systems.…”

Section: Introductionmentioning

confidence: 99%

Environment Sound Classification Based on Visual Multi-Feature Fusion and GRU-AWS

et al. 2020

View full text Add to dashboard Cite

There are two major questions regarding Environmental Sound Classification (ESC). What is the best audio recognition framework, and what is the most robust audio feature? For investigating above problems, the Gated Recurrent Unit (GRU) network was used to analyze the effect of single features such as Mel Scale Spectrogram (Mel), Log-Mel Scale Spectrogram (LM), and Mel frequency cepstral coefficient (MFCC) as well as multi-feature about Mel-MFCC, LM-MFCC, and Mel-LM-MFCC (T-M) in this paper. The experiment results show that in the ESC tasks, multi-features are better than the single features in the same dimensions, and LM-MFCC has the strongest robustness. Meanwhile, reverse sequence MFCC (R-MFCC) and forward and reverse mixed sequence MFCC (FR-MFCC) were also proposed to study the effects of sequence changes on audio. The experiment results show that the sequence transformation of audio features has less influence on the recognition tasks. Furthermore, to investigate the ESC task further we introduced the attention weight similar model (AWS) in to the multi-feature. The AWS model allows different audio feature attention weights of the same sound to learn from each other. It makes the GRU-AWS model focus on the frame-level features more effectively. The experiment results show that the GRU-AWS gets excellent performance with a recognition rate of 94.3%, and it outperforms other state-of-the-art methods.

show abstract

AdderNet: Do We Really Need Multiplications in Deep Learning?

Cited by 154 publications

References 28 publications

COVID-19 Detection Through Transfer Learning Using Multimodal Imaging Data

COVID-19 Detection Through Transfer Learning Using Multimodal Imaging Data

Bipolar Morphological Neural Networks: Gate-Efficient Architecture for Computer Vision

Environment Sound Classification Based on Visual Multi-Feature Fusion and GRU-AWS

Contact Info

Product

Resources

About