Semantic Media: Political Economy Perspectives on Platformized Fact Production

Abstract

Large media platforms are now in the habit of providing facts in their products and representing knowledge to various publics. For example, Google’s Knowledge Graph is a database of facts that Google uses to provide quick answers to publics who use their products, while Wikipedia has a product called Wikidata that similarly stores facts about the world in data formats through which various apps can retrieve the data. Microsoft, Amazon, and IBM use similar fact storing and retrieval techniques in their products. This panel introduces papers that take a political economy perspective on such platformaized versions of fact production and examines the underlying infrastructures, histories, and modeling techniques used in such knowledge representation systems. Knowledge representation, long a central topic in archiving work in library and information sciences, is a key feature of platforms and practiced by internet companies more broadly. Much of this work has historically centered on metadata models that seek to organize and describe information in standardized ways. In the context of expanding this data organizing and labeling work into the wider web, one of the main facilitators was the “Semantic Web” project proposed by Tim-Berners Lee and the World Wide Web Consortium (W3C). Today, many of the same principles, technologies, and standards that were proposed by those early projects in metadata modeling from groups like W3C are found at companies like Google and Facebook, organizations like Wikipedia, government portals, and beyond. These platform metadata models are typically produced by industry professionals (e.g., taxonomists, ontologists, knowledge engineers, etc.) who help structure information for algorithmic processing on platforms and their recommender systems. Such structured information is supposed to add a layer of contextual expressivity to web data that would otherwise be more difficult to parse, though the issue of context control is not unproblematic in relation to statements of facts. In many of these automated systems, metadata models contribute to articulating ready-made facts that then travel through these systems and eventually reach the products that are engaged by everyday web users. This panel connects scholars working in information, media studies, and science and technology studies to discuss these semantic technologies. The first paper presents data gathered from interviews with semantic web practitioners who build or have built metadata models at large internet and platform companies. It presents results from a qualitative study of these platform data management professionals (collectively referred to as “metadata modelers”) and draws from unstructured interviews (n=10) and archival research. The paper describes the image of a metadata ecology along with selected work-related contestations expressed by interview subjects regarding some of the difficulties and intractable problems in metadata modeling work. The paper includes a discussion of the political economy of platform semantics through an examination of critical semantic web literature and ends with some policy concerns. The second paper translates the method of tracing “traveling facts” from science studies to the context of online knowledge about evolving, historic events. The goal is to understand the socio-political impact of the semantic web as it has been implemented by monopolistic digital platforms and how such practices intersect in the context of Wikipedia, where the majority of knowledge graph entities are sourced from. The paper describes how the adoption (and domination) by platform companies of linked data has catalyzed a re-shaping of web content to accord with the question and answer linked data formats, weakening the power of open content licenses to support local knowledge and consolidating the power of algorithmic knowledge systems that favor knowledge monopolies. The third paper discusses building a semantic foundation for machine learning and examines how information infrastructures that convey meaning are intimately tied to colonial labor relations. It traces the practice of building a digital infrastructure that enables machines to learn from human language. The paper describes examples from an ethnographic study of semantic computing and its infrastructuring practices to show how such techniques are materially and discursively performative in their co-emergence with techno-epistemic discourses and politico-economic structures. It examines sociomaterial process in which classifications, standards, metadata, and methods co-emerge with processes of signification that reconstitute and/or shift hegemonic ecologies of knowledge. The fourth paper evaluates and examines the ethics of “free” data (CC-0) in Wikidata by evaluating the sources and usage of data from and within Wikidata. From knowledge graphs to AI training, Wikidata is the semantic web platform that is being used across the Internet to power new platforms. Through a consideration of the ways in which Wikidata scrapes Wikipedia’s “share alike” knowledge through scraping metadata and the significant donations and partnerships from large technology firms (Google in particular), this paper addresses ethical concerns within the largest semantic web platform, how these transformations of knowledge alienate donated volunteer labor, and offers some ways in which these issues might be mitigated.

Shaping information and knowledge on climate change technologies: A cross‐country qualitative analysis of carbon capture and storage results on Google search

“…In addition to the information boxes, Google highlights some sources in the section "People also ask" and featured snippets by presenting information extracted from them directly on the SERP page (Iliadis, 2022). These sources appear to be selected based on how well Google matches content to the specific query, often presenting information in the form of Q&A.…”

“…Their mediating role makes search engines a central element in the production, distribution, presentation, and accessibility of information across the world. They not only hierarchize social relations and entities (Iliadis, 2022) but also influence what people know or ignore in different moments and contexts, ultimately shaping the knowledge produced on topics such as environmental issues or new and emerging technologies (Eklöf & Mager, 2013;Haider & Rödl, 2023). This is particularly relevant because internet sources are increasingly used as legitimate sources of information on scientific and technological issues (Oltra et al, 2014).…”

“…The way information is displayed on the page is particularly relevant since users seldom look beyond the first few results (Höchstötter & Lewandowski, 2009). Google's recent move to provide facts, suggest questions, and present data directly on the results page reinforces this risk, as it allows users to conduct "zeroclick searches," where they get answers to their queries straight on Google's results page (Iliadis, 2022).…”

“…Similarly, extracting URLs often means that, except for the ranking list, the way information is visually presented in the Search Engines Interface is not considered in the analysis. This is a limitation because, as we have seen, Google increasingly extracts information from websites and presents it on its page through information boxes, suggested questions, and featured snippets (Iliadis, 2022;Schultheiß & Lewandowski, 2021). These elements of the SERP interface are central to how people interact with and interpret information through Google search engines (Höchstötter & Lewandowski, 2009;Schultheiß & Lewandowski, 2021).…”

Search Fluency Mistaken for Understanding: Ease of Information Retrieval from the Internet Inflates Internal Knowledge Confidence

“…Getting content into Featured Snippets is a desirable content optimization strategy as featured snippets increase the likelihood that a Google user does not click on any other search result. These forms of semantic media—technologies that orchestrate and convey “knowledge” about things directly in media products (Iliadis, 2022)—allow users to resolve a query more quickly and easily without having to visit multiple sites to gather information.…”