PF-TL: Payload Feature-Based Transfer Learning for Dealing with the Lack of Training Data

Abstract

The number of studies on applying machine learning to cyber security has increased over the past few years. These studies, however, are facing difficulties with making themselves usable in the real world, mainly due to the lack of training data and reusability of a created model. While transfer learning seems like a solution to these problems, the number of studies in the field of intrusion detection is still insufficient. Therefore, this study proposes payload feature-based transfer learning as a solution to the lack of training data when applying machine learning to intrusion detection by using the knowledge from an already known domain. Firstly, it expands the extracting range of information from header to payload to accurately deliver the information by using an effective hybrid feature extraction method. Secondly, this study provides an improved optimization method for the extracted features to create a labeled dataset for a target domain. This proposal was validated on publicly available datasets, using three distinctive scenarios, and the results confirmed its usability in practice by increasing the accuracy of the training data created from the transfer learning by 30%, compared to that of the non-transfer learning method. In addition, we showed that this approach can help in identifying previously unknown attacks and reusing models from different domains.

EmSM: Ensemble Mixed Sampling Method for Classifying Imbalanced Intrusion Detection Data

“…The data distribution was changed by subjecting the strings to vectorization when features were extracted from the payload segments to change the data distribution. The sampling techniques that were used were custom [35], term frequency-inverse document frequency (TF-IDF), hash, and Word2vec string vectors. As shown in Figure 10, the performance results depend on the type of string vector.…”

“…First, the training dataset is preprocessed before resampling. For the features of the intrusion detection dataset, we select hybrid features extracted from the domain and attack features according to the process of Jung et al [35]. First, the dataset with imbalanced classes is subjected to preprocessing for feature extraction based on intrusion detection characteristics.…”

“…Features were extracted using the keywords associated with the attack types for the separated fields: http_url, http_query, and http_body. The features were then categorized, and string vectorization was conducted [35].…”

Deep Learning Methods in Web Intrusion Detection: A Systematic Review

“…Trending research that follow non-classical methods to overcome one or more of the previously mentioned challenges are studied in the literature. Use of GANs and Transfer learning to help in learning phase in case of lack of attack samples can be found in [47], [44], [45].…”

“…Transfer learning methods proved to be successful in the fields of NLP, and visual recognition, and recently, IDS models. In [44] authors focused on Handling 2 known problems: Lack of training data and re-usability of created models.The model executes Payload featurebased transfer learning by using the knowledge from an already known domain and create a labelled dataset for a target domain.The source domain is well known attack signatures data. Authors claims the model increases the accuracy of the training data created from the transfer learning by 30% compared to non-transfer learning methods.…”