The selection of layers in the transfer learning fine‐tuning process ensures a pre‐trained model's accuracy and adaptation in a new target domain. However, the selection process is still manual and without clearly defined criteria. If the wrong layers in a neural network are selected and used, it could lead to poor accuracy and model generalization in the target domain. This paper introduces the use of Kullback–Leibler divergence on the weight correlations of the model's convolutional neural network layers. The approach identifies the positive and negative weights in the ImageNet initial weights selecting the best‐suited layers of the network depending on the correlation divergence. We experiment on four publicly available datasets and six ImageNet pre‐trained models used in past studies for results comparisons. This proposed approach method yields better accuracies than the standard fine‐tuning baselines with a margin accuracy rate of 10.8%–24%, thereby leading to better model adaptation for target transfer learning tasks.
In Kenya, the number of fatalities from road accidents rise year after year due to various causes. However, these numbers differ year after year and it is very difficult to identify the causation making analysis and management of anti-accident public campaigns difficult. With the use of Bayesian networks, the causal analysis can be probabilistically estimated giving a better analysis and therefore better measures in addressing the underlying causes. This paper utilises data from the Kenya National Transport Safety Authority website which is pre-processed and prepared for use in a Bayesian network model. Thereafter a Bayesian network model is built using 70% of the dataset as the training data and 30% as testing data. The model is developed with the aid of the Weka software utilising a sample of 120 instances from the prepared data with 401 instances. Furthermore, to validate the model, a Naïve Bayes model is developed with the same dataset. The Bayesian network model results in 69.125% accuracy which is lower compared to those given by the naïve Bayes model with 72.5% accuracy possibly due to the fact that Naïve Bayes algorithm performs well even with small amounts of data. Also, from the results, the model identifies that most of the accidents are driver related with 63.8% on the Bayesian network and 78.2% on the Naïve Bayes model and therefore more need to be done in addressing the driver causes. However, more variables need to be introduced in the dataset by the transport agency.
Adapting the target dataset for a pre-trained model is still challenging. These adaptation problems result from a lack of adequate transfer of traits from the source dataset; this often leads to poor model performance resulting in trial and error in selecting the best performing pre-trained model. This paper introduces the conflation of source domain low-level textural features extracted using the first layer of the pretrained model. The extracted features are compared to the conflated low-level features of the target dataset to select a higher quality target dataset for improved pre-trained model performance and adaptation. From comparing the various probability distance metrics, Kullback-Leibler is adopted to compare the samples from both domains. We experiment on three publicly available datasets and two ImageNet pre-trained models used in past studies for results comparisons. This proposed approach method yields two categories of the target samples with those with lower Kullback-Leibler values giving better accuracy, precision and recall. The samples with the lower Kullback-Leibler values give a higher margin accuracy rate of 6.21% to 7.27%, thereby leading to better model adaptation for target transfer learning datasets and tasks
The selection of layers in the transfer learning fine-tuning process ensures a pre-trained model’s accuracy and adaptation in a new target domain. However, the selection process is still manual and without clearly defined criteria. If the wrong layers in a neural network are selected and used, it could lead to poor accuracy and model generalisation in the target domain. This paper introduces the use of Kullback-Leibler divergence on the weight correlations of the model’s convolutional neural network layers. The approach identifies the positive and negative weights in the ImageNet initial weights selecting the best-suited layers of the network depending on the correlation divergence. We experiment on four publicly available datasets and four ImageNet pre-trained models that have been used in past studies for results comparisons. This proposed approach method yields better accuracies than the standard fine-tuning baselines with a margin accuracy rate of 10.8% to 24%, thereby leading to better model adaptation for target transfer learning tasks.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
customersupport@researchsolutions.com
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.