SemifreddoNets: Partially Frozen Neural Networks for Efficient Computer Vision Systems

Isikdogan, Leo F.; Nayak, Bhavin V.; Wu, Chyuan-Tyng; Moreira, Joao Peralta; Rao, Sushma; Michael, Gilad

doi:10.1007/978-3-030-58583-9_12

Cited by 5 publications

(2 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…,6 to focus on situations with few annotations. We also investigate whether freezing the learned parameters f Θ is a viable option, as this reduces the training effort and the number of required annotated samples [27]. Since 2% available annotations are difficult but not impossible to solve if the pretext task provides good representations, we choose to select exemplary results for each pretext task for this annotation rate.…”

Section: ) Pretext Comparisonmentioning

confidence: 99%

Self-Supervised Learning for Annotation Efficient Biomedical Image Segmentation

Rettenberger

Schilling

Elser

et al. 2023

IEEE Trans. Biomed. Eng.

View full text Add to dashboard Cite

The scarcity of high-quality annotated data is omnipresent in machine learning. Especially in biomedical segmentation applications, experts need to spend a lot of their time into annotating due to the complexity. Hence, methods to reduce such efforts are desired. Methods: Self-Supervised Learning (SSL) is an emerging field that increases performance when unannotated data is present. However, profound studies regarding segmentation tasks and small datasets are still absent. A comprehensive qualitative and quantitative evaluation is conducted, examining SSL's applicability with a focus on biomedical imaging. We consider various metrics and introduce multiple novel application-specific measures. All metrics and state-of-the-art methods are provided in a directly applicable software package (https://osf.io/gu2t8/). Results: We show that SSL can lead to performance improvements of up to 10%, which is especially notable for methods designed for segmentation tasks. Conclusion: SSL is a sensible approach to data-efficient learning, especially for biomedical applications, where generating annotations requires much effort. Additionally, our extensive evaluation pipeline is vital since there are significant differences between the various approaches. Significance: We provide biomedical practitioners with an overview of innovative data-efficient solutions and a novel toolbox for their own application of new approaches. Our pipeline for analyzing SSL methods is provided as a ready-to-use software package.

show abstract

Section: ) Pretext Comparisonmentioning

confidence: 99%

Self-Supervised Learning for Annotation Efficient Biomedical Image Segmentation

Rettenberger

Schilling

Elser

et al. 2023

IEEE Trans. Biomed. Eng.

View full text Add to dashboard Cite

show abstract

“…Moreover, Isikogan et al [35] introduced a novel approach for freezing specific parameters within each layer, substituting multipliers with fixed scalers and replacing the network with optimized full-pipeline hardware blocks. The proposed network organization offers a balance between flexibility and cost because the weights can be configured at various scales and levels of abstraction, distinguishing it from conventional layer-by-layer freezing technique.…”

Section: Weight Transfer and Freezingmentioning

confidence: 99%

SFAO: Sign-Flipping-Aware Optimization for Early-Stopping of Binarized Neural Networks

Kang,

Ryu,

Kang

et al. 2023

IEEE Access

View full text Add to dashboard Cite

One of the vital challenges for the binary neural networks (BNNs) is improving their inference performance by expanding their data representation capabilities for figuring out delicate patterns and nuances in the data. Addressing the explosive computational demands on neural network training is essential to guarantee sustainable development and scalable deployment. However, mitigating the increase in the computational cost during the training phase is critical for ensuring sustainability and scalability during deployment. In this study, an advanced sign-flipping-aware optimizer (SFAO) that focuses on BNNs was introduced to diminish the computational burden. SFAO balanced the model performance and computational cost through sign-flipping-aware updating rules throughout the training of BNNs. SFAO optimizer, tailored for BNNs with binary weight-specific updating rules, considerably reduced the computing resources needed for training on the CIFAR-10 dataset. Specifically, it surpassed the conventional full-precision updating rule by reducing the total instruction count by 21.89%. In contrast, SFAO showed a marginal 0.44% decline in the image classification accuracy relative to the updating rules for the full-precision parameters. Furthermore, the implementation of early stopping using the sign flip rate led to a notable reduction of 9.37% in the average computation time per network for the ImageNet dataset.INDEX TERMS artificial intelligence, model compression, optimizer, efficient machine learning, binarized neural networks, layer freezing.

show abstract

LWDN: lightweight DenseNet model for plant disease diagnosis

Dheeraj,

Chand

2024

J Plant Dis Prot

View full text Add to dashboard Cite

SemifreddoNets: Partially Frozen Neural Networks for Efficient Computer Vision Systems

Cited by 5 publications

References 14 publications

Self-Supervised Learning for Annotation Efficient Biomedical Image Segmentation

Self-Supervised Learning for Annotation Efficient Biomedical Image Segmentation

SFAO: Sign-Flipping-Aware Optimization for Early-Stopping of Binarized Neural Networks

LWDN: lightweight DenseNet model for plant disease diagnosis

Contact Info

Product

Resources

About