Analysis of Transfer Learning Performance Measures

Weiss, Karl R.; Khoshgoftaar, Taghi M.

doi:10.1109/iri.2017.43

Cited by 7 publications

(3 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, even though the performance gap is both data-and algorithm-dependent, the metric is considered crucial for a more informative and finer generalization bound. The study by Weiss and Khoshgoftaar [14] provides a discussion of the relative performance analysis of state-of-the-art transfer learning algorithms and traditional machine learning algorithms. Their analysis addresses the question of whether the area under the curve (AUC) performance is predictive of classification accuracy in a transfer learning environment, where there is no labeled target data to perform validation methods.…”

Section: Critical State-of-the-art Review Of Tl Performance Metricsmentioning

confidence: 99%

“…Our proposition necessitates that we briefly discuss the issues of negative transfer and catastrophic forgetting. One conclusion from [14] is that analyzing the relative performance of TL algorithms across a wide range of distortion profiles should be considered an area for future research. Negative transfer occurs when the source domain data and tasks contribute to lower learning performance in the target domain.…”

Section: Critical State-of-the-art Review Of Tl Performance Metricsmentioning

confidence: 99%

See 1 more Smart Citation

Comprehensive Sensitivity Analysis Framework for Transfer Learning Performance Assessment for Time Series Forecasting: Basic Concepts and Selected Case Studies

Kambale,

Salem,

Benarbia

et al. 2024

Symmetry

View full text Add to dashboard Cite

Recently, transfer learning has gained popularity in the machine learning community. Transfer Learning (TL) has emerged as a promising paradigm that leverages knowledge learned from one or more related domains to improve prediction accuracy in a target domain with limited data. However, for time series forecasting (TSF) applications, transfer learning is relatively new. This paper addresses the need for empirical studies as identified in recent reviews advocating the need for practical guidelines for Transfer Learning approaches and method designs for time series forecasting. The main contribution of this paper is the suggestion of a comprehensive framework for Transfer Learning Sensitivity Analysis (SA) for time series forecasting. We achieve this by identifying various parameters seen from various angles of transfer learning applied to time series, aiming to uncover factors and insights that influence the performance of transfer learning in time series forecasting. Undoubtedly, symmetry appears to be a core aspect in the consideration of these factors and insights. A further contribution is the introduction of four TL performance metrics encompassed in our framework. These TL performance metrics provide insight into the extent of the transferability between the source and the target domains. Analyzing whether the benefits of transferred knowledge are equally or unequally accessible and applicable across different domains or tasks speaks to the requirement of symmetry or asymmetry in transfer learning. Moreover, these TL performance metrics inform on the possibility of the occurrence of negative transfers and also provide insight into the possible vulnerability of the network to catastrophic forgetting. Finally, we discuss a sensitivity analysis of an Ensemble TL technique use case (with Multilayer Perceptron models) as a proof of concept to validate the suggested framework. While the results from the experiments offer empirical insights into various parameters that impact the transfer learning gain, they also raise the question of network dimensioning requirements when designing, specifically, a neural network for transfer learning.

show abstract

Section: Critical State-of-the-art Review Of Tl Performance Metricsmentioning

confidence: 99%

Section: Critical State-of-the-art Review Of Tl Performance Metricsmentioning

confidence: 99%

Comprehensive Sensitivity Analysis Framework for Transfer Learning Performance Assessment for Time Series Forecasting: Basic Concepts and Selected Case Studies

Kambale,

Salem,

Benarbia

et al. 2024

Symmetry

View full text Add to dashboard Cite

show abstract

“…This includes noise (either class or attribute), class imbalance, outliers, high-dimensionality, as well others [105][106][107]. These issues have a unique impact on transfer learning tasks as they may present in either the source or the target domain which causes changing circumstances [108][109][110].…”

Section: Comparative Analysismentioning

confidence: 99%