OmniCrawl: Comprehensive Measurement of Web Tracking With Real Desktop and Mobile Browsers

Abstract

Over half of all visits to websites now take place in a mobile browser, yet the majority of web privacy studies take the vantage point of desktop browsers, use emulated mobile browsers, or focus on just a single mobile browser instead. In this paper, we present a comprehensive web-tracking measurement study on mobile browsers and privacy-focused mobile browsers. Our study leverages a new web measurement infrastructure, OmniCrawl, which we develop to drive browsers on desktop computers and smartphones located on two continents. We capture web tracking measurements using 42 different non-emulated browsers simultaneously. We find that the third-party advertising and tracking ecosystem of mobile browsers is more similar to that of desktop browsers than previous findings suggested. We study privacy-focused browsers and find their protections differ significantly and in general are less for lower-ranked sites. Our findings also show that common methodological choices made by web measurement studies, such as the use of emulated mobile browsers and Selenium, can lead to website behavior that deviates from what actual users experience.

AutoFR: Automated Filter Rule Generation for Adblocking

“…AutoFR can also be applied to other platforms, such as mobile, smart TVs, and VR devices, as there is need for better platform-specific filter lists, in terms of coverage and breakage [72,82,84]. On mobile and Smart TVs specifically, we could leverage existing tools to automatically explore apps or mobile browsers [17,50,72,84]. Finally, AutoFR can also be used to maintain and improve existing filter lists over time, not only to generate them from scratch.…”

Analysing and strengthening OpenWPM's reliability

“…Equivalent amount of ads/trackers traffic. To assess the amount of trackers and advertisers in traffic, we use the same approach as previous works [5,14,41]: use the EasyList and EasyPrivacy blocklists 10 to identify trackers. Our results show that WPM and WPM hide encounter a near equal rate of advertisers and trackers.…”

“…This relies on an (often unstated) assumption that the data as collected is representative of what a regular, human-controlled browser would encounter. Previous works [5,14,39,41] have shown that this is not always the case: websites have been found to omit content (advertisements, video, JavaScript execution, login forms, etc.) or require completion of a CAPTCHA for automated clients.…”

“…We find several new ways in which a malicious website can attack OpenWPM's data recording. While both cloaking and data recording attacks are possible, [14] finds that Selenium-based bots receive far less third-party traffic, which indicates cloaking happens in practice. We develop a stealth extension for OpenWPM to prevent cloaking and test its performance on sites where OpenWPM-detectors were found.…”

“…In their study, OpenWPM clients encountered three times more tracking domains and had more interaction with third-party domains than human-controlled browsers. Cassel et al [14] investigate the reliability of emulated browsers. To avoid bot detection, they created their own tooling to remotely control a browser.…”