Transformer-based LLMs in Cybersecurity: An in-depth Study on Log Anomaly Detection and Conversational Defense Mechanisms

Balasubramanian, Prasasthy; Seby, Justin; Kostakos, Panos

doi:10.1109/bigdata59044.2023.10386976

Cited by 2 publications

(2 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hierarchical thematic classification in system logs offers a structured and efficient methodology for reducing data size and improving anomaly detection by considering the hierarchical structure of themes and the underlying semantic relationships in system logs. Hierarchical thematic classification in a system log can be an effective strategy for reducing its size and preparing it for fine-tuning Large Language Models (LLM) in anomaly detection, which are starting to yield promising results [4]. This technique involves organizing themes in a hierarchical structure that reflects the relationship between them, which can facilitate the identification of patterns and more precise anomaly detection [10,11].…”

Section: Related Workmentioning

confidence: 99%

“…Modern anomaly detection systems based on logs utilize various artificial intelligence techniques, the most commonly used being those based on machine-learning algorithms [3] or natural language processing like Large Language Models [4]. These techniques consist of three phases: log parsing, feature extraction, and anomaly detection using a trained model.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Semantic Hierarchical Classification Applied to Anomaly Detection Using System Logs with a BERT Model

Corbelle,

Carneiro,

Cacheda

2024

Applied Sciences

View full text Add to dashboard Cite

The compaction and structuring of system logs facilitate and expedite anomaly and cyberattack detection processes using machine-learning techniques, while simultaneously reducing alert fatigue caused by false positives. In this work, we implemented an innovative algorithm that employs hierarchical codes based on the semantics of natural language, enabling the generation of a significantly reduced log that preserves the semantics of the original. This method uses codes that reflect the specificity of the topic and its position within a higher hierarchical structure. By applying this catalog to the analysis of logs from the Hadoop Distributed File System (HDFS), we achieved a concise summary with non-repetitive themes, significantly speeding up log analysis and resulting in a substantial reduction in log size while maintaining high semantic similarity. The resulting log has been validated for anomaly detection using the “bert-base-uncased” model and compared with six other methods: PCA, IM, LogCluster, SVM, DeepLog, and LogRobust. The reduced log achieved very similar values in precision, recall, and F1-score metrics, but drastically reduced processing time.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%