Enhancing Network Attack Detection Accuracy through the Integration of Large Language Models and Synchronized Attention Mechanism

Abstract

In this study, we propose a novel method for detecting cyberattack behaviors by leveraging the combined strengths of large language models and a synchronized attention mechanism. Extensive experiments conducted on diverse datasets, including server logs, financial behaviors, and comment data, demonstrate the significant advantages of this method over existing models such as Transformer, BERT, OPT-175B, LLaMa, and ChatGLM3-6B in key performance metrics such as precision, recall, and accuracy. For instance, on the server log dataset, the method achieved a precision of 93%, a recall of 91%, and an accuracy of 92%; on the financial behavior dataset, it reached a precision of 90%, a recall of 87%, and an accuracy of 89%; and on the comment data dataset, it excelled with a precision of 95%, a recall of 93%, and an accuracy of 94%. The introduction of a synchronized attention mechanism and a newly designed synchronized loss function proved especially effective, enhancing the method’s ability to process multi-source data and providing superior performance in identifying complex cyberattack patterns. Ablation experiments further validated the crucial roles of these innovations in boosting model performance: the synchronous attention mechanism substantially improved the model’s precision, recall, and accuracy to 93%, 89%, and 91% respectively, far exceeding other attention mechanisms. Similarly, the synchronized loss showcased a significant advantage, achieving the best performance across all tested metrics compared to traditional cross-entropy loss, focal loss, and MSE. These results underscore the method’s ability to deeply mine and analyze semantic information and contextual relationships within text data as well as to effectively integrate and process multimodal data, thereby offering strong technical support for the accurate and efficient detection of cyberattack behaviors.