SoK: The Challenges, Pitfalls, and Perils of Using Hardware Performance Counters for Security

Abstract

Hardware Performance Counters (HPCs) have been available in processors for more than a decade. These counters can be used to monitor and measure events that occur at the CPU level. Modern processors provide hundreds of hardware events that can be monitored, and with each new processor architecture more are added. Yet, there has been little in the way of systematic studies on how performance counters can best be utilized to accurately monitor events in real-world settings. Especially when it comes to the use of HPCs for security applications, measurement imprecisions or incorrect assumptions regarding the measured values can undermine the offered protection.To shed light on this issue, we embarked on a year-long effort to (i) study the best practices for obtaining accurate measurement of events using performance counters, (ii) understand the challenges and pitfalls of using HPCs in various settings, and (iii) explore ways to obtain consistent and accurate measurements across different settings and architectures. Additionally, we then empirically evaluated the way HPCs have been used throughout a wide variety of papers. Not wanting to stop there, we explored whether these widely used techniques are in fact obtaining performance counter data correctly. As part of that assessment, we (iv) extended the seminal work of Weaver and McKee from almost 10 years ago on non-determinism in HPCs, and applied our findings to 56 papers across various application domains.In that follow-up study, we found the acceptance of HPCs in security applications is in stark contrast to other application areas -especially in the last five years. Given that, we studied an additional representative set of 41 works from the security literature that rely on HPCs, to better elucidate how the intricacies we discovered can impact the soundness and correctness of their approaches and conclusions. Toward that goal, we (i) empirically evaluated how failure to accommodate for various subtleties in the use of HPCs can undermine the effectiveness of security applications, specifically in the case of exploit prevention and malware detection. Lastly, we showed how (ii) an adversary can manipulate HPCs to bypass certain security defenses.

pAElla: Edge AI-Based Real-Time Malware Detection in Data Centers

“…Instead, when using IF with performance metrics (+ coarse-grain power statistics) we observed a high F1-score, with 0 % of FA rate, in the system in idle, HPCG, and QE, while we obtained a poor F1-score for Gromacs and HPL, and a really low F1-score with high FA rate for NPB btC9, and NPB btC16. When looking at the overall F1-score, we obtain a low value of 0.758, which is in line with results in [34], [35], when using tree-based models with performance counters for malware detection (reason why they do not advice this technique for security purposes). Finally, in the case of AE, we obtain poor scores for all benchmarks, except for HPCG, NPB btC9, and NPB btC16, and an overall F1-score of 0.627.…”

Dynamic Malware Analysis in the Modern Era—A State of the Art Survey

“…The two comparison papers mentioned above [88,90] point out the weak connection between HPC measurements and the high-level code executed by malware, but the same conclusion can be drawn by side-channel analysis using external hardware. The conclusion is that side-channel analysis is only as strong as the statistical correlation between two separate views: high-level code and low-level hardware.…”

“…Using HPCs to detect malware has been tested by various researchers with inconclusive results. One study [88] showed that while it is possible to use HPCs to detect ROP (return-oriented programming) execution, HPCs are less effective for malware detection. ROP is a common technique used by malware to counter anti-exploit mechanisms, which is based on reusing compiled code that was previously loaded into the victim's RAM.…”

“…An additional review paper [88] surveyed over 50 publications that cover the topic of HPC. The paper also includes an experiment comparing ROP detection and malware detection.…”

Detecting (absent) app-to-app authentication on cross-device short-distance channels

“…Moreover, this technology is used to establish the so-called mobile ad-hoc clouds, which take advantage of unused resources of nearby devices to provide cloud services, such as data and computation offloading. This is also a typical case of IoT environment, where IoT devices communicate with each other on short-distance channels [10].…”