An Experimental Study of Diversity with Off-the-Shelf AntiVirus Engines

Abstract

Abstract-Fault tolerance in the form of diverse redundancy is well known to improve the detection rates for both malicious and non-malicious failures. What is of interest to designers of security protection systems are the actual gains in detection rates that they may give. In this paper we provide exploratory analysis of the potential gains in detection capability from using diverse AntiVirus products for the detection of self-propagating malware. The analysis is based on 1599 malware samples collected by the operation of a distributed honeypot deployment over a period of 178 days. We sent these samples to the signature engines of 32 different AntiVirus products taking advantage of the VirusTotal service. The resulting dataset allowed us to perform analysis of the effects of diversity on the detection capability of these components as well as how their detection capability evolves in time.

Does Malware Detection Improve with Diverse AntiVirus Products? An Empirical Study

“…And just as experimentation in the natural sciences is supported by laboratories, experimentation for a science of cybersecurity will require test beds where controlled experiments can be run." In this paper we present results of an empirical study about possible benefits of diversity with currently spreading malware and compare our findings with those reported in [6][7][8]. The main aim of our study is to verify the extent to which the findings previously reported are relevant with more recent malware.…”

“…The results presented in [6][7][8] are intriguing. However, they concern a specific snapshot in the detection capabilities of AVs against malware threats prevalent in that time period: 1599 malware samples collected from a distributed honeypot deployment over a period of 178 days from February to August 2008.…”

“…Two of the authors of this paper have previously reported [6][7][8] results from a study on the detection capabilities of different AVs and potential improvements in detection that can be observed from using diverse AVs. In those studies we reported that some AVs achieved high detection rates, but none detected all the malware samples.…”

“…In the security field the threat landscape changes rapidly and it is not clear to what extent these findings can be generalized to currently spreading malware. It is also not clear whether the diversity benefits reported in [6][7][8] are specific to that specific collection environment and time period, or whether they are consistent with other environments and time periods.…”

“…The rest of the paper is organized as follows: Section 2 summarizes related work; Section 3 describes the data collection infrastructure; Section 4 presents the results of our study; Section 5 compares the results reported in this paper with the ones in [6][7][8]; Section 6 discusses the implications of our results on the decision making about security protection systems, and presents conclusions and possible further work.…”