PolyLoss: A Polynomial Expansion Perspective of Classification Loss Functions

Leng, Zhaoqi; Tan, Mingxing; Liu, Chenxi; Cubuk, Ekin D.; Shi, X.; Cheng, Shuyang; Anguelov, Dragomir

doi:10.48550/arxiv.2204.12511

Cited by 24 publications

(30 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous studies have shown that [25] the first polynomial contributes more than half of the gradient in the later stage of training, highlighting the importance of the first polynomial. Better segmentation accuracy can be obtained by simply adjusting the first coefficient of the polynomial, without the need to adjust all polynomial coefficients 𝛼 𝑗 .…”

Section: Poly-dicelossmentioning

confidence: 95%

“…Loss function is any differentiable function that maps predictions and labels to scalars. Existing studies have shown that [25] the accuracy of classification can be improved by applying the cross-entropy loss function to the Taylor expansion and adjusting the polynomial coefficient. Based on the loss function diceloss, which is commonly used in medical images, this paper proposed a loss function framework named poly-diceloss.…”

Section: Poly-dicelossmentioning

confidence: 99%

See 1 more Smart Citation

Semantic Segmentation of Nasal Septum Based on Parameter-Free Attention U-Net

Cui

Wang

et al. 2023

Preprint

View full text Add to dashboard Cite

Accurate segmentation of nasal septum plays a key role in assisting doctors in nasal surgery. However, this practice is still a great challenge due to the variety in the shapes of nasal septum of different people. This paper brought forward an effective parameter-free attention U-Net for accurate segmentation of nasal septum. This attention module is an energy function, which is used to identify the importance of each pixel and provide three-dimensional attention weight for feature map inference in the layer without any additional parameters. On this basis, a new loss function of poly-diceloss was introduced, which regarded the diceloss as a linear combination of polynomial functions and significantly improved the segmentation performance by introducing a super parameter. In this paper, a data set named “nasal septum” was constructed. Based on this data set, a comparison was made with the most advanced network and the experimental results showed that the indexes of SAMU-Net proposed in this paper were: \(TPR\) 98.47%, \(PPV\) 83.57%, \(JAC\) 82.37%, \(Dice\) 90.23%, respectively, using the least network parameters.

show abstract

Section: Poly-dicelossmentioning

confidence: 95%

Section: Poly-dicelossmentioning

confidence: 99%

Semantic Segmentation of Nasal Septum Based on Parameter-Free Attention U-Net

Cui

Wang

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…This advance avoids problems of model saturation and overfitting that traditional CNN encounters. Although different optimization techniques, such as dense connection and fine-tuning, are applied to further improve the model performance [47][48][49][50], they rest upon these building block and milestone developments of these CNN models.…”

Section: • Image-level Classificationmentioning

confidence: 99%

GeoAI for Large-Scale Image Analysis and Machine Vision: Recent Progress of Artificial Intelligence in Geography

Hsu

2022

IJGI

View full text Add to dashboard Cite

GeoAI, or geospatial artificial intelligence, has become a trending topic and the frontier for spatial analytics in Geography. Although much progress has been made in exploring the integration of AI and Geography, there is yet no clear definition of GeoAI, its scope of research, or a broad discussion of how it enables new ways of problem solving across social and environmental sciences. This paper provides a comprehensive overview of GeoAI research used in large-scale image analysis, and its methodological foundation, most recent progress in geospatial applications, and comparative advantages over traditional methods. We organize this review of GeoAI research according to different kinds of image or structured data, including satellite and drone images, street views, and geo-scientific data, as well as their applications in a variety of image analysis and machine vision tasks. While different applications tend to use diverse types of data and models, we summarized six major strengths of GeoAI research, including (1) enablement of large-scale analytics; (2) automation; (3) high accuracy; (4) sensitivity in detecting subtle changes; (5) tolerance of noise in data; and (6) rapid technological advancement. As GeoAI remains a rapidly evolving field, we also describe current knowledge gaps and discuss future research directions.

show abstract

“…Important for our purposes, NCE+RCE has been successfully employed in computer vision applications, particularly in the very high noise regime. Overall, α-loss, focal loss, and NCE+RCE have all been shown to be robust to label noise in the training data, and hence comprise a strong representative subset of the robust loss function literature (for more examples, see [26], [27], [28], [29]). However, to the best of our knowledge, each of these loss functions have not been previously considered in the joint setting of training and test domain shift, which we argue is the real-world scenario addressed by our proposed AUGLOSS methodology.…”

Section: Loss Functions For Noisy Labelingmentioning

confidence: 99%

AugLoss: A Learning Methodology for Real-World Dataset Corruption

Otstot¹,

Cava²,

Sypherd³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep Learning (DL) models achieve great successes in many domains. However, DL models increasingly face safety and robustness concerns, including noisy labeling in the training stage and feature distribution shifts in the testing stage. Previous works made significant progress in addressing these problems, but the focus has largely been on developing solutions for only one problem at a time. For example, recent work has argued for the use of tunable robust loss functions to mitigate label noise, and data augmentation (e.g., AUGMIX) to combat distribution shifts. As a step towards addressing both problems simultaneously, we introduce AUGLOSS, a simple but effective methodology that achieves robustness against both train-time noisy labeling and test-time feature distribution shifts by unifying data augmentation and robust loss functions. We conduct comprehensive experiments in varied settings of real-world dataset corruption to showcase the gains achieved by AUGLOSS compared to previous state-of-the-art methods. Lastly, we hope this work will open new directions for designing more robust and reliable DL models under real-world corruptions. The GitHub link to the paper and code repository is: https://github.com/SankarLab/AugLoss.

show abstract

PolyLoss: A Polynomial Expansion Perspective of Classification Loss Functions

Cited by 24 publications

References 27 publications

Semantic Segmentation of Nasal Septum Based on Parameter-Free Attention U-Net

Semantic Segmentation of Nasal Septum Based on Parameter-Free Attention U-Net

GeoAI for Large-Scale Image Analysis and Machine Vision: Recent Progress of Artificial Intelligence in Geography

AugLoss: A Learning Methodology for Real-World Dataset Corruption

Contact Info

Product

Resources

About