How biased is the sample? Reverse engineering the ranking algorithm of Facebook’s Graph application programming interface

Abstract

Facebook research has proliferated during recent years. However, since November 2017, Facebook has introduced a new limitation on the maximum amount of page posts retrievable through their Graph application programming interface, while there is limited documentation on how these posts are selected. This paper compares two datasets of the same Facebook page, a full dataset obtained before the introduction of the limitation and a partial dataset obtained after, and employs bootstrapping technique to assess the bias caused by the new limitation. This paper demonstrates that posts with high user engagement, Photo posts and Video posts, are over-represented, while Link posts are underrepresented. Top-term analysis reveals that there are significant differences in the most prominent terms between the full and partial dataset. This paper also reverse engineered the new application programming interface's ranking algorithm to identify the features of a post that would affect its odds of being selected. Sentiment analysis reveals that there are significant differences in the sentiment word usage between the selected and non-selected posts. This paper has significant implications for the representativeness of research that use Facebook page data collected after the introduction of the limitation.

What do we want from Explainable Artificial Intelligence (XAI)? – A stakeholder perspective on XAI and a conceptual model guiding interdisciplinary XAI research

“…Gain scientific insights from the system User [5,7,12,14,35,37,44,47,60,63,66,67,70,87,97,100,106] Security Assess and increase a system's security [13,16,25,26,29,33,34,37,39,44,46,49,55,57,62,63,67,74,75,78,80,92,94,107,108] [ 28,31,43,50] Trust Calibrate appropriate trust in the system User, Deployer…”

Understanding Hong Kong nationalism: A topic network approach

“…2 The data collection for Youngspiration was supplemented with data collected by Chung-hong Chan and King-wa Fu due to Facebook's restriction on the Graph API (cf. Ho, 2020) 3 While the translations were not perfect, it is the best available solution deal to the vast quantity. All the analysis was conducted with original Chinese text, and therefore the error in the translation will not affect the estimations below.…”

API Governance: The Case of Facebook’s Evolution

“…1 However, even minor API changes can have significant research implications. Newly imposed data limitations may introduce potential biases that undermine the representativeness of data studies (Ho, 2020). Such API-based studies can arguably only be interpreted and replicated alongside historical information about how the APIs used have changed and evolved.…”