On building machine learning pipelines for Android malware detection: a procedural survey of practices, challenges and opportunities

Abstract

As the smartphone market leader, Android has been a prominent target for malware attacks. The number of malicious applications (apps) identified for it has increased continually over the past decade, creating an immense challenge for all parties involved. For market holders and researchers, in particular, the large number of samples has made manual malware detection unfeasible, leading to an influx of research that investigate Machine Learning (ML) approaches to automate this process. However, while some of the proposed approaches achieve high performance, rapidly evolving Android malware has made them unable to maintain their accuracy over time. This has created a need in the community to conduct further research, and build more flexible ML pipelines. Doing so, however, is currently hindered by a lack of systematic overview of the existing literature, to learn from and improve upon the existing solutions. Existing survey papers often focus only on parts of the ML process (e.g., data collection or model deployment), while omitting other important stages, such as model evaluation and explanation. In this paper, we address this problem with a review of 42 highly-cited papers, spanning a decade of research (from 2011 to 2021). We introduce a novel procedural taxonomy of the published literature, covering how they have used ML algorithms, what features they have engineered, which dimensionality reduction techniques they have employed, what datasets they have employed for training, and what their evaluation and explanation strategies are. Drawing from this taxonomy, we also identify gaps in knowledge and provide ideas for improvement and future work.

Android Malware Detection using HexCode Features

“…An essential and intricate aspect of organizing malware analysis lies in the comprehensive consideration of all aspects and attributes of Android applications, while safeguarding the retention of critical features. Several recent review papers provide comprehensive overviews of past works and research efforts in Android malware analysis [8][9][10][11]. This assumes paramount significance and complexity, notably due to increase in use of obfuscation techniques.…”

Machine Learning and Deep Learning Based Model for the Detection of Rootkits Using Memory Analysis

“…ML algorithms can analyze memory dumps and recognize patterns associated with malware, rootkits, and other malicious activities. it has been used for over a past decade to detect the malwares [5]. By training on labeled data, ML models can learn to identify anomalies and deviations from normal system behavior [6].…”

Hybrid Android Malware Detection: A Review of Heuristic-Based Approach

“…However, despite this surge in research activity, most of the review papers that are cited do not primarily focus on examining hybrid approaches. For example, Shu et al [20], Koushki et al [21], and Meijin et al [22] review papers include discussions on malware detection that involve static, dynamic, and hybrid approaches. It is crucial to have a focused review that considers the distinct characteristics of hybrid approaches compared to other approaches, such as static and dynamic approaches.…”