Feature modeling and cluster analysis of malicious Web traffic

Abstract

Feature modeling and cluster analysis of malicious Web traffic by Ana Dimitrijevikj Many attackers find Web applications to be attractive targets since they are widely used and have many vulnerabilities to exploit. The goal of this thesis is to study patterns of attacker activities on typical Web based systems using four data sets collected by honeypots, each in duration of almost four months. The contributions of our work include cluster analysis and modeling the features of the malicious Web traffic. Some of our main conclusions are: (1) Features of malicious sessions, such as Number of Requests, Bytes Transferred, and Duration, follow skewed distributions, including heavy-tailed. (2) Number of requests per unique attacker follows skewed distributions, including heavy-tailed, with a small number of attackers submitting most of the malicious traffic. (3) Cluster analysis provides an efficient way to distinguish between attack sessions and vulnerability scan sessions. First, I would like to thank my committee chair and advisor, Dr. Katerina Goseva-Popstojanova, for her guidance, support and encouragement throughout my graduate studies. Also, I would like to thank Dr. James Mooney and Dr. Arun Ross for being my graduate committee members. I am grateful for the support and advice from all my graduate committee members and I am thankful for their collaboration. I would like to acknowledge that my work has been funded by the National Science Foundation under CAREER grant CNS-0447715. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation. I also want to thank and acknowledge Risto Pantev, Brandon S. Miller, J. Alex Baker, Jonathan Lynch, and David Krovich for their collaboration in the research project. I want to thank all my friends for their help and support. Finally, I would like to express my deepest gratitude to my mother, father, and brother. They all motivated and encouraged me to pursue this degree. They were always supporting me and that means the world to me. iv Contents Acknowledgements iii List of Figures vi List of Tables viii

Analysis and Classification of Current Trends in Malicious HTTP Traffic

“…For our previous work [7], [48], [50] and for the work presented here and in [2] a highinteraction honeypots were developed and deployed. These high-interaction honeypots run real services and real Web applications following the example of GenII honeypots used by the Honeynet Project [90].…”

“…Since 2008 when our first honeypot system was deployed, we collected huge amounts of data. The data that we analyzed in our previous work [7], [48], [50], and for the work presented here and in [2] was for time periods where honeypots had minimal or no downtime. In total we managed to create four datasets from the observed malicious HTTP traffic from the advertised honeypots.…”

“…For this and for our previous work [48] and [50], as well as for the work presented in [2] and [7] we collected Web server logs from the two most commonly used Web servers Microsoft IIS and Apache [56] running on the two most commonly used server operating systems i.e. Windows and Linux.…”

“…This work together with [2] and [7] is a part of larger effort aimed at Improving Web Quality through an Integrated Approach [12]. Over a period of several years our research group deployed several honeypots with different configurations to collect malicious traffic.…”

Classification of Malicious Web Traffic

“…The popularity of these applications and their frequent exploitation motivated us to analyze attackers activities on Web systems running Web 2.0 applications. For this purpose, over a period of several years, our research group [18], [55], [48] developed and deployed three high-interaction honeypots, each consisting of a three-tier Web architecture (i.e. Web server, application server, and a database server).…”