A Comparison of Loss Weighting Strategies for Multi task Learning in Deep Neural Networks

Gong, Ting; Lee, Tyler; Stephenson, Cory; Renduchintala, Venkata; Padhy, Suchismita; Ndirango, Anthony; Keskin, Gokce; Elibol, Oğuz H.

doi:10.1109/access.2019.2943604

Cited by 90 publications

(65 citation statements)

References 12 publications

(22 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…47 However, these two methods achieved poor results owing to the fact that they considered only either loss intensity or task difficulty. 48 To overcome the heterogeneity (also called in-homogeneity) 49 of the hippocampus, T1-weighted MRIs with and without contrast were adopted in the training phase. Experimental results showed that using multimodality MRIs surpassed using single-modality MRIs by a significant margin in terms of Dice, HD, and AVD, which suggested that multimodality MRIs provided complementary contextual information.…”

Section: Discussionmentioning

confidence: 99%

Automatic segmentation of hippocampus in hippocampal sparing whole brain radiotherapy: A multitask edge‐aware learning

Qiu

Yang

et al. 2021

Medical Physics

View full text Add to dashboard Cite

This study aimed to improve the accuracy of the hippocampus segmentation through multitask edge-aware learning. Method: We developed a multitask framework for computerized hippocampus segmentation. We used three-dimensional (3D) U-net as our backbone model with two training objectives: (a) to minimize the difference between the targeted binary mask and the model prediction; and (b) to optimize an auxiliary edge-prediction task which is designed to guide the model detection of the weak boundary of the hippocampus in model optimization. To balance the multiple task objectives, we proposed an improved gradient normalization by adaptively adjusting the weight of losses from different tasks. A total of 247 T1-weighted MRIs including 131 without contrast and 116 with contrast were collected from 247 patients to train and validate the proposed method. Segmentation was quantitatively evaluated with the dice coefficient (Dice), Hausdorff distance (HD), and average Hausdorff distance (AVD). The 3D U-net was used for baseline comparison. We used a Wilcoxon signed-rank test to compare repeated measurements (Dice, HD, and AVD) by different segmentations. Results: Through fivefold cross-validation, our multitask edge-aware learning achieved Dice of 0.8483 AE 0.0036, HD of 7.5706 AE 1.2330 mm, and AVD of 0.1522 AE 0.0165 mm, respectively. Conversely, the baseline results were 0.8340 AE 0.0072, 10.4631 AE 2.3736 mm, and 0.1884 AE 0.0286 mm, respectively. With a Wilcoxon signed-rank test, we found that the differences between our method and the baseline were statistically significant (P < 0.05). Conclusion:Our results demonstrated the efficiency of multitask edge-aware learning in hippocampus segmentation for hippocampal sparing whole-brain radiotherapy. The proposed framework may also be useful for other low-contrast small organ segmentations on medical imaging modalities.

show abstract

Section: Discussionmentioning

confidence: 99%

Automatic segmentation of hippocampus in hippocampal sparing whole brain radiotherapy: A multitask edge‐aware learning

Qiu

Yang

et al. 2021

Medical Physics

View full text Add to dashboard Cite

show abstract

“…This means human knowledge on the application subject and on the various task sharing methods is a necessity to find the best method for the application. It has been found that MTL is unlikely to improve performance unless the tasks and the weighting strategies are carefully chosen ( Gong et al., 2019 ). Continued research is needed for optimal strategies to choose and balance tasks.…”

Section: Methods To Integrate Human Knowledgementioning

confidence: 99%

Integrating Machine Learning with Human Knowledge

et al. 2020

View full text Add to dashboard Cite

Summary Machine learning has been heavily researched and widely used in many disciplines. However, achieving high accuracy requires a large amount of data that is sometimes difficult, expensive, or impractical to obtain. Integrating human knowledge into machine learning can significantly reduce data requirement, increase reliability and robustness of machine learning, and build explainable machine learning systems. This allows leveraging the vast amount of human knowledge and capability of machine learning to achieve functions and performance not available before and will facilitate the interaction between human beings and machine learning systems, making machine learning decisions understandable to humans. This paper gives an overview of the knowledge and its representations that can be integrated into machine learning and the methodology. We cover the fundamentals, current status, and recent progress of the methods, with a focus on popular and new topics. The perspectives on future directions are also discussed.

show abstract

“…To achieve this, we designed the custom loss function to optimize the ratio between accuracy and under-estimations. The intuition behind the design of the custom loss function stems from multi task learning [16], we used the weighted average of the two loss functions used in previous work [25]. Our objective is to increase the accuracy of our predictions while maintaining a low level of under-estimations.…”

Section: Introductionmentioning

confidence: 99%

Predicting file lifetimes for data placement in multi-tiered storage systems for HPC

ThomasLuis

GougeaudSebastien

RubiniStéphane

et al. 2021

SIGOPS Oper. Syst. Rev.

View full text Add to dashboard Cite

The emergence of Exascale machines in HPC will have the foreseen consequence of putting more pressure on the storage systems in place, not only in terms of capacity but also bandwidth and latency. With limited budget we cannot imagine using only storage class memory, which leads to the use of a heterogeneous tiered storage hierarchy. In order to make the most efficient use of the high performance tier in this storage hierarchy, we need to be able to place user data on the right tier and at the right time. In this paper, we assume a 2-tier storage hierarchy with a high performance tier and a high capacity archival tier. Files are placed on the high performance tier at creation time and moved to capacity tier once their lifetime expires (that is once they are no more accessed). The main contribution of this paper lies in the design of a file lifetime prediction model solely based on its path based on the use of Convolutional Neural Network. Results show that our solution strikes a good trade-off between accuracy and under-estimation. Compared to previous work, our model made it possible to reach an accuracy close to previous work (around 98.60% compared to 98.84%) while reducing the underestimations by almost 10x to reach 2.21% (compared to 21.86%). The reduction in underestimations is crucial as it avoids misplacing files in the capacity tier while they are still in use.

show abstract

A Comparison of Loss Weighting Strategies for Multi task Learning in Deep Neural Networks

Cited by 90 publications

References 12 publications

Automatic segmentation of hippocampus in hippocampal sparing whole brain radiotherapy: A multitask edge‐aware learning

Automatic segmentation of hippocampus in hippocampal sparing whole brain radiotherapy: A multitask edge‐aware learning

Integrating Machine Learning with Human Knowledge

Predicting file lifetimes for data placement in multi-tiered storage systems for HPC

Contact Info

Product

Resources

About