The case for voter-centered audits of search engines during political elections

Mustafaraj, Eni; Lurie, Emma; Devine, Claire

doi:10.1145/3351095.3372835

Cited by 17 publications

(12 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This review corroborates the suggestion by Mustafaraj et al [120] that existing audit literature has only accounted for a narrow set of use cases. Future audits should strive for user-centered audits, and may benefit from scoping to specific use cases.…”

Section: Remaining Work: Distortionsupporting

confidence: 89%

“…Future audits should strive for user-centered audits, and may benefit from scoping to specific use cases. For example, such audits can build on the work of Lai and Luczak-Roesch [97] which scoped to public officials' use of search algorithms, and the work of Mustafaraj et al [120] which outlines methods for voter-centric audits. Also, Fischer et al [59] scoped their audit to local news, a topic which deserves more audit attention given the current crisis in local journalism and its threat to impacted communities (e.g.…”

Section: Remaining Work: Distortionmentioning

confidence: 99%

See 1 more Smart Citation

Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits

Bandy

2021

Preprint

View full text Add to dashboard Cite

While algorithm audits are growing rapidly in commonality and public importance, relatively little scholarly work has gone toward synthesizing prior work and strategizing future research in the area. This systematic literature review aims to do just that, following PRISMA guidelines in a review of over 500 English articles that yielded 62 algorithm audit studies. The studies are synthesized and organized primarily by behavior (discrimination, distortion, exploitation, and misjudgement), with codes also provided for domain (e.g. search, vision, advertising, etc.), organization (e.g. Google, Facebook, Amazon, etc.), and audit method (e.g. sock puppet, direct scrape, crowdsourcing, etc.). The review shows how previous audit studies have exposed public-facing algorithms exhibiting problematic behavior, such as search algorithms culpable of distortion and advertising algorithms culpable of discrimination. Based on the studies reviewed, it also suggests some behaviors (e.g. discrimination on the basis of intersectional identities), domains (e.g. advertising algorithms), methods (e.g. code auditing), and organizations (e.g. Twitter, TikTok, LinkedIn) that call for future audit attention. The paper concludes by offering the common ingredients of successful audits, and discussing algorithm auditing in the context of broader research working toward algorithmic justice.

show abstract

Section: Remaining Work: Distortionsupporting

confidence: 89%

Section: Remaining Work: Distortionmentioning

confidence: 99%

Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits

Bandy

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…This may present a real obstacle to finding proper medical care, as it has been shown that young adults have difficulty finding the correct keywords to find accurate information about emergency contraception (Hargittai & Young, 2012). More research is needed to understand how someone might formulate a query specific to finding abortion services, perhaps through ethnographic or survey-based studies (Mustafaraj et al, 2020;Trielli & Diakopoulos, 2020;Tripodi, 2018) , or through data sharing agreements with search engines, but it is clear that promoting digital literacy may be an effective way to promote access to quality information and increase healthcare utilization. Second, when the search results were returned, they were likely to reflect the availability of ACs in the state of the query origin, as well as the demographic inequalities that especially disadvantage those searching from lower-income and less populated areas inequalities which have long been observed (Fried, 2000).…”

Section: User Implicationsmentioning

confidence: 99%

“…Systematic study of these issues will hopefully identify locales, such as Joplin, Missouri, which may suffer from the perceived lack of health access, despite services being available in the vicinity. In the light of the ongoing debate over how search engines personalize content around political issues (Introna & Nissenbaum, 2000;Robertson, 2018), it would be interesting to perform the current study while logged into accounts associated with differing political stances (Hannak et al, 2013;Le et al, 2019), or by using queries generated by people from across the political spectrum (Mustafaraj et al, 2020;Trielli & Diakopoulos, 2020).…”

Section: Policy Implicationsmentioning

confidence: 99%

Googling for Abortion: Search Engine Mediation of Abortion Accessibility in the United States

Mejova

Gracyk

Robertson

2022

JQD

View full text Add to dashboard Cite

Among the myriad barriers to abortion access, crisis pregnancy centers (CPCs) pose an additional difficulty by targeting women with unexpected or “crisis” pregnancies in order to dissuade them from the procedure. Web search engines may prove to be another barrier, being in a powerful position to direct their users to health information, and above all, health services. In this study we ask, to what degree does Google Search provide quality responses to users searching for an abortion provider, specifically in terms of directing them to abortion clinics (ACs) or CPCs. To answer this question, we considered the scenario of a woman searching for abortion services online, and conducted 10 abortion-related queries from 467 locations across the United States once a week for 14 weeks. Overall, among Google’s location results that feature businesses alongside a map, 79.4% were ACs, and 6.9% were CPCs. When an AC was returned, it was the closest known AC location 86.9% of the time. However, when a CPC appeared in a result set, it was the closest one to the search location 75.9% of the time. Examining correlates of AC results, we found that fewer AC results were returned for searches from poorer and rural areas, and those with TRAP laws governing AC facility and clinician requirements. We also observed that Google’s performance on our queries significantly improved following a major algorithm update. These results have important implications concerning health access quality and equity, both for individual users and public health policy.

show abstract

“…Previous audit studies have investigated the search engines for partisan bias [48,56], gender bias [10,39], content diversity [52,61,62], and price discrimination [33]. However, only a few have systematically investigated search engines' role in surfacing misinformation ( [36] is the only exception).…”

Section: Search Engine Auditsmentioning

confidence: 99%

Auditing E-Commerce Platforms for Algorithmically Curated Vaccine Misinformation

Juneja

Mitra

2021

Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

There is a growing concern that e-commerce platforms are amplifying vaccine-misinformation. To investigate, we conduct two-sets of algorithmic audits for vaccine misinformation on the search and recommendation algorithms of Amazon-world's leading eretailer. First, we systematically audit search-results belonging to vaccine-related search-queries without logging into the platformunpersonalized audits. We find 10.47% of search-results promote misinformative health products. We also observe ranking-bias, with Amazon ranking misinformative search-results higher than debunking search-results. Next, we analyze the effects of personalization due to account-history, where history is built progressively by performing various real-world user-actions, such as clicking a product. We find evidence of filter-bubble effect in Amazon's recommendations; accounts performing actions on misinformative products are presented with more misinformation compared to accounts performing actions on neutral and debunking products. Interestingly, once user clicks on a misinformative product, homepage recommendations become more contaminated compared to when user shows an intention to buy that product. CCS CONCEPTS• Information systems → Personalization; Content ranking; Web crawling; • Human-centered computing → Human computer interaction (HCI).

show abstract

The case for voter-centered audits of search engines during political elections

Cited by 17 publications

References 31 publications

Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits

Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits

Googling for Abortion: Search Engine Mediation of Abortion Accessibility in the United States

Auditing E-Commerce Platforms for Algorithmically Curated Vaccine Misinformation

Contact Info

Product

Resources

About