Incorporating Transfer Learning in CNN Architecture

Abstract

Machine learning (ML) is a data intensive process. For training of ML algorithms huge datasets are required.There are times when enough data is not available due to multitude of reasons. This could be due to lack ofavailability of annotated data in a particular domain or paucity of time in data collection process resulting innon-availability of enough data. Many a times data collection is very expensive and in few domains data collectionis very difficult. In such cases, if methods can be designed to reuse the knowledge gained in one domain havingenough training data, to some other related domain having less training data, then problems associated with lackof data can be overcome. Transfer Learning (TL) is one such method. TL improves the performance of the targetdomain through knowledge transfer from some different but related source domain. This knowledge transfer canbe in form of feature extraction, domain adaptation, rule extraction for advice and so on. TL also works withvarious kinds of ML tasks related to supervised, unsupervised and reinforcement learning. The ConvolutionalNeural Networks are well suited for the TL approach. The general features learned on a base network (source)are shifted to the target network. The target network then uses its own data and learns new features specific toits requirement.