An ensemble of pre-trained transformer models for imbalanced multiclass malware classification

Demirkıran, Ferhat; Çayır, Aykut; Ünal, Uğur; Dağ, Hasan

doi:10.1016/j.cose.2022.102846

Cited by 21 publications

(6 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…From Table 4, as can be observed, the first five methods give a baseline for classification tasks, and models based on malware images or traditional NLP methods can achieve accuracy around 0.90. The model of [21] achieved an accuracy of 0.90, the model of [48] achieved an accuracy of 0.93, and our model had an accuracy of 0.96. From a baseline perspective, all three models go beyond other basic or advanced methods and achieve a better result, demonstrating the effectiveness of the API calls and Encoder/Transformer architecture models.…”

Section: Comparison With Previous Methodsmentioning

confidence: 69%

“…The first five methods are classic methods [14,[44][45][46][47] to do the malware family classification, and we report the results from their papers. The following five methods [16,20,21,23,48] are the latest effective work on the classification based on API calls, so we reproduce the methods and offer a convincing comparison result. The [21] method adopts a two-way feature extraction architecture for API calls, but the core module is a multi-layer CNN, and the correlation analysis is performed through Bi-LSTM.…”

Section: Comparison With Previous Methodsmentioning

confidence: 99%

“…Our architecture is unified based on Transformer and attention mechanisms, and so by comparing with this method, it can reflect the role of the backbone network. The study [48] further adopted a pre-training mechanism and integrated multiple Transformer architecture models through Random Forest, which has similarities with our backbone network. Moreover, it uses the process of integrating Random Forest, which is like Step 2.…”

Section: Comparison With Previous Methodsmentioning

confidence: 99%

See 2 more Smart Citations

TTDAT: Two-Step Training Dual Attention Transformer for Malware Classification Based on API Call Sequences

Wang,

Lin,

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

The surge in malware threats propelled by the rapid evolution of the internet and smart device technology necessitates effective automatic malware classification for robust system security. While existing research has primarily relied on some feature extraction techniques, issues such as information loss and computational overhead persist, especially in instruction-level tracking. To address these issues, this paper focuses on the nuanced analysis of API (Application Programming Interface) call sequences between the malware and system and introduces TTDAT (Two-step Training Dual Attention Transformer) for malware classification. TTDAT utilizes Transformer architecture with original multi-head attention and an integrated local attention module, streamlining the encoding of API sequences and extracting both global and local patterns. To expedite detection, we introduce a two-step training strategy: ensemble Transformer models to generate class representation vectors, thereby bolstering efficiency and adaptability. Our extensive experiments demonstrate TTDAT’s effectiveness, showcasing state-of-the-art results with an average F1 score of 0.90 and an accuracy of 0.96.

show abstract

Section: Comparison With Previous Methodsmentioning

confidence: 69%

Section: Comparison With Previous Methodsmentioning

confidence: 99%

Section: Comparison With Previous Methodsmentioning

confidence: 99%

See 1 more Smart Citation

TTDAT: Two-Step Training Dual Attention Transformer for Malware Classification Based on API Call Sequences

Wang,

Lin,

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…We compiled and analyzed these results as reference data, and the comparative outcomes are illustrated in Table 4. From Table 4, Demirkıran et al [43] tested multiple pre-trained models, including BERT, CA-NINE-S, and their proposed Random Transformer Forest (RTF) model, on the mal-api-2019 dataset. Li et al [44] utilized RNNs and transformer structures to classify malware by learning interactive features in API call sequences.…”

Section: Baselines Comparative Evaluationmentioning

confidence: 99%

Channel Features and API Frequency-Based Transformer Model for Malware Identification

Qian,

Cong

2024

Sensors

View full text Add to dashboard Cite

Malicious software (malware), in various forms and variants, continues to pose significant threats to user information security. Researchers have identified the effectiveness of utilizing API call sequences to identify malware. However, the evasion techniques employed by malware, such as obfuscation and complex API call sequences, challenge existing detection methods. This research addresses this issue by introducing CAFTrans, a novel transformer-based model for malware detection. We enhance the traditional transformer encoder with a one-dimensional channel attention module (1D-CAM) to improve the correlation between API call vector features, thereby enhancing feature embedding. A word frequency reinforcement module is also implemented to refine API features by preserving low-frequency API features. To capture subtle relationships between APIs and achieve more accurate identification of features for different types of malware, we leverage convolutional neural networks (CNNs) and long short-term memory (LSTM) networks. Experimental results demonstrate the effectiveness of CAFTrans, achieving state-of-the-art performance on the mal-api-2019 dataset with an F1 score of 0.65252 and an AUC of 0.8913. The findings suggest that CAFTrans improves accuracy in distinguishing between various types of malware and exhibits enhanced recognition capabilities for unknown samples and adversarial attacks.

show abstract

“…Pre-trained transformer models, such as BERT, have become popular, offering fine-tuning for tasks like question answering, text classification, and language generation (Casola et al, 2022;Min et al, 2021;Qiu et al, 2020). These models have also found applications in cybersecurity, including cyberbullying detection, malware classification, and API call-based malware detection (Demirkiran et al, 2022;Oak et al, 2019;Paul & Saha, 2022).…”

Section: Salo Et Al Introduce a Hybrid Dimensionality Reduction Techn...mentioning

confidence: 99%

Network intrusion detection system by learning jointly from tabular and text‐based features

Düzgün,

Çayır,

Ünal

et al. 2023

Expert Systems

Self Cite

View full text Add to dashboard Cite

Network intrusion detection systems (NIDS) play a critical role in maintaining the security and integrity of computer networks. These systems are designed to detect and respond to anomalous activities that may indicate malicious intent or unauthorized access. The need for robust NIDS solutions has never been more pressing in today's digital landscape, characterized by constantly evolving cyber threats. Deploying effective NIDS can be challenging, particularly in accurately identifying network anomalies amid the ever‐increasing sophisticated and difficult‐to‐detect cyber threats. The motivation for our research stems from the recognition that while NIDS studies have made significant strides, there remains a crucial need for more effective and accurate methods to detect network anomalies. Commonly used features in NIDS studies include network logs, with some studies exploring text‐based features such as payload. However, traditional machine and deep learning models may need to be improved in learning jointly from tabular and text‐based features. Here, we present a new approach that integrates both tabular and text‐based features to improve the performance of NIDS. Our research aims to address the existing limitations of NIDS and contribute to the development of more reliable and efficient network security solutions by introducing more effective and accurate methods for detecting network anomalies. Our internal experiments have revealed that the deep learning approach utilizing tabular features produces favourable results, whereas the pre‐trained transformer approach needs to perform sufficiently. Hence, our proposed approach, which integrates both feature types using deep learning and pre‐trained transformer approaches, achieves superior performance. These findings indicate that integrating both feature types using deep learning and pre‐trained transformer approaches can significantly improve the accuracy of network anomaly detection. Moreover, our proposed approach outperforms the state‐of‐the‐art methods in terms of accuracy, F1‐score, and recall on commonly used NIDS datasets consisting of ISCX‐IDS2012, UNSW‐NB15, and CIC‐IDS2017, with F1‐scores of 99.80%, 92.37%, and 99.69%, respectively, indicating its effectiveness in detecting network anomalies.

show abstract

An ensemble of pre-trained transformer models for imbalanced multiclass malware classification

Cited by 21 publications

References 29 publications

TTDAT: Two-Step Training Dual Attention Transformer for Malware Classification Based on API Call Sequences

TTDAT: Two-Step Training Dual Attention Transformer for Malware Classification Based on API Call Sequences

Channel Features and API Frequency-Based Transformer Model for Malware Identification

Network intrusion detection system by learning jointly from tabular and text‐based features

Contact Info

Product

Resources

About