Beyond the Hype: An Evaluation of Commercially Available Machine Learning–based Malware Detectors

Abstract

There is a lack of scientific testing of commercially available malware detectors, especially those that boast accurate classification of never-before-seen (i.e., zero-day) files using machine learning (ML). Consequently, efficacy of malware detectors is opaque, inhibiting end users from making informed decisions and researchers from targeting gaps in current detectors. In this paper, we present a scientific evaluation of four prominent commercial malware detection tools to assist an organization with two primary questions: To what extent do ML-based tools accurately classify previously and never-before-seen files? Is purchasing a network-level malware detector worth the cost? To investigate, we tested each tool against 3,536 total files (2,554 or 72% malicious, 982 or 28% benign) of a variety of file types, including hundreds of malicious zero-days, polyglots, and APT-style files, delivered on multiple protocols. We present statistical results on detection time and accuracy, consider complementary analysis (using multiple tools together), and provide two novel applications of the recent cost-benefit evaluation procedure of Iannacone & Bridges. Although the ML-based tools are more effective at detecting zero-day files and executables, the signature-based tool might still be an overall better option. Both network-based tools provide substantial (simulated) savings when paired with either host tool, yet both show poor detection rates on protocols other than HTTP or SMTP. Our results show that all four tools have near-perfect precision but alarmingly low recall, especially on file types other than executables and office files—37% of malware, including all polyglot files, were undetected. Priorities for researchers and takeaways for end users are given. Code for future use of the cost model is provided.

Comprehensive Analysis of Endpoint Security Strategies, Technologies and Challenges

“…The proposed technique, known as MAAC, use semantics to decrease the volume of cautions and furnish security investigators with a positioned rundown of potential assault ways and dubious hosts. MAAC demonstrated powerful in decreasing copy alarms and reducing the focal point of safety examiners to basic situations, showing easing the difficulties of ready over-burden in endpoint security potential [1]. Taking everything into account, the field of endpoint security innovations keeps on progressing, with different methodologies and systems created to address the advancing danger scene.…”

“…Endpoint security arrangements with VPNs, multifaceted confirmation, and secure access controls empowered associations to get remote workplaces. Contextual investigations flourish where these actions forestalled unapproved access and information breaks while guaranteeing the efficiency of remote employees [1]. 6.…”

Cryptocurrency Mining Malware Detection Based on Behavior Pattern and Graph Neural Network

“…Traditional malware detection techniques mainly rely on signature matching techniques and heuristic detection, but this type of rule matching has the low capability for generalization and finding unknown malware [5]. Machine learning techniques have evolved in recent years and have become another important way to solve malware detection.…”