Population automation:  An interview with Wikipedia bot pioneer Ram-Man

Livingstone, Randall M.

doi:10.5210/fm.v21i1.6027

Cited by 7 publications

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are thousands of bots in Wikipedia, and tasks that they are delegated extend to every level of the encyclopedia and the community who writes it. Bots have existed almost from the beginning of Wikipedia's 15-year history, starting with Ram-Bot in 2002, which almost doubled the size of Wikipedia by creating an article about every city and town in the U.S. from public domain census data (Livingstone, 2016). A suite of different bots, maintained by different contributors, automatically remove edits that they determine to be spam, vandalism, plagiarism, or gibberish (Geiger and Halfaker, 2013).…”

Section: Generalizing the Ethnographic Vignettes To The Rest Of Wikip...mentioning

confidence: 99%

Beyond opening up the black box: Investigating the role of algorithmic systems in Wikipedian organizational culture

Geiger¹

2017

Big Data & Society

View full text Add to dashboard Cite

Scholars and practitioners across domains are increasingly concerned with algorithmic transparency and opacity, interrogating the values and assumptions embedded in automated, black-boxed systems, particularly in user-generated content platforms. I report from an ethnography of infrastructure in Wikipedia to discuss an often understudied aspect of this topic: the local, contextual, learned expertise involved in participating in a highly automated social-technical environment. Today, the organizational culture of Wikipedia is deeply intertwined with various data-driven algorithmic systems, which Wikipedians rely on to help manage and govern the ''anyone can edit'' encyclopedia at a massive scale. These bots, scripts, tools, plugins, and dashboards make Wikipedia more efficient for those who know how to work with them, but like all organizational culture, newcomers must learn them if they want to fully participate. I illustrate how cultural and organizational expertise is enacted around algorithmic agents by discussing two autoethnographic vignettes, which relate my personal experience as a veteran in Wikipedia. I present thick descriptions of how governance and gatekeeping practices are articulated through and in alignment with these automated infrastructures. Over the past 15 years, Wikipedian veterans and administrators have made specific decisions to support administrative and editorial workflows with automation in particular ways and not others. I use these cases of Wikipedia's bot-supported bureaucracy to discuss several issues in the fields of critical algorithms studies; critical data studies; and fairness, accountability, and transparency in machine learning-most principally arguing that scholarship and practice must go beyond trying to ''open up the black box'' of such systems and also examine sociocultural processes like newcomer socialization.

show abstract

Section: Generalizing the Ethnographic Vignettes To The Rest Of Wikip...mentioning

confidence: 99%

Beyond opening up the black box: Investigating the role of algorithmic systems in Wikipedian organizational culture

Geiger¹

2017

Big Data & Society

View full text Add to dashboard Cite

show abstract

“…Like the rest of Wikipedia's system of processes, quality control policy and practice are open to redesign via a consensus 6 conversation. Historically, the people with the skills and inclination to develop software tools that support work processes in Wikipedia have held a large amount of power in deciding what types of work will and will not be supported [10,27,31,34,40]. In theory, one promising strategy to change quality control practices is to develop tools that capture an alternative vision of what's important (e.g.…”

Section: The Problem: Stagnation In Quality Control Practicesmentioning

confidence: 99%

“…Without such a tight feedback loop, we most likely would not have noticed how poorly ORES's damage detection models were performing in practice. Worse, it might have caused vandal fighters to be increasingly (and inappropriately) skeptical of contributions by anonymous editors and newly registered editors-two groups of contributors that are already met with unnecessary hostility 27 [20].…”

Section: Bias Against Anonymous Editorsmentioning

confidence: 99%

ORES: Lowering Barriers with Participatory Machine Learning in Wikipedia

Halfaker,

Geiger

2019

Preprint

View full text Add to dashboard Cite

Algorithmic systems-from rule-based bots to machine learning classifiers-have a long history of supporting the essential work of content moderation and other curation work in peer production projects. From counter-vandalism to task routing, basic machine prediction has allowed open knowledge projects like Wikipedia to scale to the largest encyclopedia in the world, while maintaining quality and consistency. However, conversations about how quality control should work and what role algorithms should play have generally been led by the expert engineers who have the skills and resources to develop and modify these complex algorithmic systems. In this paper, we describe ORES: an algorithmic scoring service that supports real-time scoring of wiki edits using multiple independent classifiers trained on different datasets. ORES decouples several activities that have typically all been performed by engineers: choosing or curating training data, building models to serve predictions, auditing predictions, and developing interfaces or automated agents that act on those predictions. This meta-algorithmic system was designed to open up socio-technical conversations about algorithmic systems in Wikipedia to a broader set of participants. In this paper, we discuss the theoretical mechanisms of social change ORES enables and detail case studies in participatory machine learning around ORES from the 4 years since its deployment.

show abstract

“…We are Wikipedia researchers who have been working on various issues around bots and automation for many years, and we were surprised and intrigued to hear these findings. While bot-bot conflict certainly takes place in Wikipedia for a variety of reasons, conflict at the size, scale, and severity the authors of the EGBF paper claimed has not been mentioned in almost a decade of multi-disciplinary scholarship on the governance of Wikipedia bots (and particularly the Bot Approvals Group) [13,19,24,34,43,55,65]. Previous literature has largely celebrated the successes of Wikipedia's approach to automation, which is based on the same principles as editing Wikipedia articles: decentralized consensus-building, scaffolded by formalized policies and processes.…”

Section: Even Good Bots Fight? the Broader Implications Of Wikipedianmentioning

confidence: 99%

“…As bots play a substantial role in Wikipedia, they have been extensively discussed in previous literature [13,24,34,43,55,62,65], even incidentally in the context of other topics [8,19,28,51]. It is important to distinguish between: 1) bot-bot conflict, in which automated software agents get into "edit wars" with each other over the content of articles due to being programmed with opposing directives and 2) conflict about bots, in which human Wikipedians conflict with each other about what kinds of tasks ought to be automated in Wikipedia and how.…”

Section: Bot Governancementioning

confidence: 99%

Operationalizing Conflict and Cooperation between Automated Software Agents in Wikipedia

Geiger

Halfaker

2017

Proc. ACM Hum.-Comput. Interact.

View full text Add to dashboard Cite

This paper replicates, extends, and refutes conclusions made in a study published in PLoS ONE ("Even Good Bots Fight"), which claimed to identify substantial levels of conflict between automated software agents (or bots) in Wikipedia using purely quantitative methods. By applying an integrative mixed-methods approach drawing on trace ethnography, we place these alleged cases of bot-bot conflict into context and arrive at a better understanding of these interactions. We found that overwhelmingly, the interactions previously characterized as problematic instances of conflict are typically better characterized as routine, productive, even collaborative work. These results challenge past work and show the importance of qualitative/quantitative collaboration. In our paper, we present quantitative metrics and qualitative heuristics for operationalizing bot-bot conflict. We give thick descriptions of kinds of events that present as bot-bot reverts, helping distinguish conflict from non-conflict. We computationally classify these kinds of events through patterns in edit summaries. By interpreting found/trace data in the socio-technical contexts in which people give that data meaning, we gain more from quantitative measurements, drawing deeper understandings about the governance of algorithmic systems in Wikipedia. We have also released our data collection, processing, and analysis pipeline, to facilitate computational reproducibility of our findings and to help other researchers interested in conducting similar mixed-method scholarship in other platforms and contexts.

show abstract

Population automation: An interview with Wikipedia bot pioneer Ram-Man

Cited by 7 publications

References 0 publications

Beyond opening up the black box: Investigating the role of algorithmic systems in Wikipedian organizational culture

Beyond opening up the black box: Investigating the role of algorithmic systems in Wikipedian organizational culture

ORES: Lowering Barriers with Participatory Machine Learning in Wikipedia

Operationalizing Conflict and Cooperation between Automated Software Agents in Wikipedia

Contact Info

Product

Resources

About