MODIS Green-Tide Detection With a Squeeze and Excitation Oriented Generative Adversarial Network

Jin, Xiao; Li, Yun; Wan, Jianhua; Lyu, Xinrong; Ren, Peng; Shang, Junliang

doi:10.1109/access.2022.3180331

Cited by 4 publications

(1 citation statement)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We noticed that this invention of the ResNets cause to return to rise of exponentially increased number of parameters in deeper networks and the increased pressure to introduce the stringent reduction of the parameters. The Squeeze-and-Excitation (SE) attention technique [13][14][15][16], introduced by Hu et al, offers a channel-wise recalibration to enhance model accuracy without significant computational overhead.…”

Section: Introductionmentioning

confidence: 99%

A Speed-up Channel Attention Technique for Accelerating the Learning Curve of a Binarized Squeeze-and-Excitation (SE) Based ResNet Model

Shaoqing,

Yamauchi

2024

JAIT

View full text Add to dashboard Cite

The use of 1-bit representation for network weights, as opposed to the conventional 32-bit, has been investigated to save on the required power and memory footprint. Squeeze-and-Excitation (SE) based channel attention techniques aim to further reduce the number of parameters by eliminating redundant channels. However, this approach leads to a significant drawback of an unstable and slow learning curve, especially when compared to fitting parameters in SE networks. To address this issue, this paper presents the first attempt to accelerate the learning curve, even with a 1-bit representation for weights across the entire Squeeze-and-Excitation Residual Network (SEResNet14). The proposed technique within the SE module significantly speeds up channel attention, yielding a steeper learning curve for the network. We also extensively investigate the impact of activation functions within the SE module, aiming to understand their performance-enhancing attributes when applied with the proposed technique. Experimental results demonstrate that even under stringent compression, an appropriate choice of activation function can still ensure the efficacy of our technique in the SE module. We found that the proposed technique results in: (1) a 60% reduction in the required number of epochs to achieve an error rate of 0.3; and (2) a decrease in the error rate by approximately 44% at the 10th epoch, compared to a baseline method that does not use the proposed scheme.

show abstract

Section: Introductionmentioning

confidence: 99%