Mind the Gap: On Bridging the Semantic Gap Between Machine Learning and Malware Analysis.

Smith, Michael R.; Johnson, Nicholas T.; Ingram, Joey Burton; Carbajal, Armida J.; Haus, Bridget Irene; Domschot, Eva; Ramyaa, Ramyaa; Lamb, Christopher R.; Verzi, Stephen Joseph; Kegelmeyer, W. Philip

doi:10.2172/1831009

Cited by 9 publications

(9 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For training, we consider a subset of the 600K-sample Ember training dataset, as well as the full Brazilian dataset; we train a separate MalConv model for each case. Our subsets consist of 10-30K samples, which is consistent with the dataset sizes that many malware research groups currently work with [44]. While we would have preferred to experiment with augmenting more datasets, we are constrained since many existing datasets do not contain raw binaries as mentioned in 2.…”

Section: Discussionmentioning

confidence: 99%

Marvolo: Programmatic Data Augmentation for Practical ML-Driven Malware Detection

Wong¹,

Raff²,

Holt³

et al. 2022

Preprint

View full text Add to dashboard Cite

Data augmentation has been rare in the cyber security domain due to technical difficulties in altering data in a manner that is semantically consistent with the original data. This shortfall is particularly onerous given the unique difficulty of acquiring benign and malicious training data that runs into copyright restrictions, and that institutions like banks and governments receive targeted malware that will never exist in large quantities. We present MARVOLO, a binary mutator that programmatically grows malware (and benign) datasets in a manner that boosts the accuracy of ML-driven malware detectors. MARVOLO employs semantics-preserving code transformations that mimic the alterations that malware authors and defensive benign developers routinely make in practice , allowing us to generate meaningful augmented data. Crucially, semantics-preserving transformations also enable MARVOLO to safely propagate labels from original to newly-generated data samples without mandating expensive reverse engineering of binaries. Further, MARVOLO embeds several key optimizations that keep costs low for practitioners by maximizing the density of diverse data samples generated within a given time (or resource) budget. Experiments using wide-ranging commercial malware datasets and a recent ML-driven malware detector show that MARVOLO boosts accuracies by up to 5%, while operating on only a small fraction (15%) of the potential input binaries.Preprint. Under review.

show abstract

Section: Discussionmentioning

confidence: 99%

Marvolo: Programmatic Data Augmentation for Practical ML-Driven Malware Detection

Wong¹,

Raff²,

Holt³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Several state-of-the-art studies [10], [11], [38]- [41] using ML/DL and genetic algorithm achieve performance gains in detecting and analyzing malicious executables and mobile apps. Additionally, they have shown that automated behavioral profiling or semantic labeling for malware analysis can benefit security analysts and reduce manual work.…”

Section: B Powershell Behavioral Profilingmentioning

confidence: 99%

PowerDP: De-Obfuscating and Profiling Malicious PowerShell Commands With Multi-Label Classifiers

Tsai

Lin

et al. 2023

IEEE Access

View full text Add to dashboard Cite

In recent years, PowerShell has become the common tool that helps attackers launch targeted attacks using living-off-the-land tactics and fileless attack techniques. Unfortunately, malwarederived PowerShell Commands (PSCmds) have typically been obfuscated to hide the malicious intent from detection and analysis. Also, malicious PSCmds' expansive use of multiple obfuscation strategies and encryption methods makes them difficult to be revealed. Despite the advances in malicious PSCmds detection incorporating new approaches such as machine learning and deep learning, there is still no consensus on the solution to de-obfuscating malicious PSCmds and profiling their behavior. To address this challenge, we propose a hybrid framework that combines deep learning and program analysis for automatic PowerShell De-obfuscation and behavioral Profiling (PowerDP) through multi-label classification in a static manner. First, we use character distribution features to forecast obfuscation types of malicious PSCmds. Second, we developed an extensive de-obfuscator utilizing static regular expression replacement to recover the original content of obfuscated PSCmds based on the predicted obfuscation types. Finally, we profile the behavior of PSCmds by features extracted from the abstract syntax tree of PSCmds after deobfuscation. Our results show that PowerDP achieves a promising 99.82% accuracy and 0.18% hamming loss in obfuscation multi-label classification using deep learning. Furthermore, the successful recovery rate of the de-obfuscator against 15 obfuscation types is 98.11% on average with semantic similarity comparison, and the accuracy of the behavior multi-label classification for identifying 5 behaviors in malicious PSCmds averages 98.53%. The evaluation indicates that PowerDP is able to classify and profile complicated PSCmds.INDEX TERMS PowerShell, de-obfuscation, machine learning, deep learning, abstract syntax trees, multi-label classification, behavioral profiling 1 https://attack.mitre.org/matrices/enterprise/ documents are still the best combinations for malware delivery media. Because people remain susceptible to manipulation, human psychological weaknesses result in the main vulnerabilities that can be exploited through social engineering, e.g., spear-phishing attacks. In this scenario, attackers typically attached a well-customized malicious document containing PowerShell Commands (PSCmds) to a forged email. Additionally, impersonation is often used in sender names or contact information to lure targets into opening malicious files.Living-off-the-Land (LotL) tactics and fileless attack VOLUME 0, 2022

show abstract

“…Smith et al [115] have pointed towards the semantic gap between the machine learning and malware analysis communities. One of their proposals is to reposition the task from identifying malware to identifying behavior, making it possible to understand what a malware is doing.…”

Section: Malware Analysismentioning

confidence: 99%

Intelligent Malware Defenses

Nadeem

Rimmer

Joosen

et al. 2022

Security and Artificial Intelligence

View full text Add to dashboard Cite

With rapidly evolving threat landscape surrounding malware, intelligent defenses based on machine learning are paramount. In this chapter, we review the literature proposed in the past decade and identify the state-of-the-art in various related research directionsmalware detection, malware analysis, adversarial malware, and malware author attribution. We discuss challenges that emerge when machine learning is applied to malware. We also identify the key issues that need to be addressed by the research community in order to further deepen and systematize research in the malware domain.

show abstract

Mind the Gap: On Bridging the Semantic Gap Between Machine Learning and Malware Analysis.

Cited by 9 publications

References 0 publications

Marvolo: Programmatic Data Augmentation for Practical ML-Driven Malware Detection

Marvolo: Programmatic Data Augmentation for Practical ML-Driven Malware Detection

PowerDP: De-Obfuscating and Profiling Malicious PowerShell Commands With Multi-Label Classifiers

Intelligent Malware Defenses

Contact Info

Product

Resources

About