Diagnosing Gender Bias in Image Recognition Systems

Schwemmer, Carsten; Knight, Carly; Bello-Pardo, Emily; Oklobdzija, Stan; Schoonvelde, Martijn; Lockhart, Jeffrey W.

doi:10.31235/osf.io/as25q

Cited by 10 publications

(12 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For AI systems to be useful to society as a whole, their performance should not depend on the perceived skin tone or gender of the subjects -they should work equally well for all populations. However, current automatic vision system performance has been reported to vary with race, gender or when applied across different demographics and geographic regions (Buolamwini and Gebru, 2018;De Vries et al, 2019;Schwemmer et al, 2020). As a preliminary study assessing how Flamingo's performance varies between populations, we follow the study proposed in and report how the captioning performance of our model varies on COCO as a function of gender and race.…”

Section: Risks and Mitigation Strategiesmentioning

confidence: 95%

Flamingo: a Visual Language Model for Few-Shot Learning

Alayrac¹,

Donahue²,

Luc³

et al. 2022

Preprint

View full text Add to dashboard Cite

ordered alphabetically, † Equal contributions, ordered alphabetically, ‡ Equal senior contributions Building models that can be rapidly adapted to numerous tasks using only a handful of annotated examples is an open challenge for multimodal machine learning research. We introduce Flamingo, a family of Visual Language Models (VLM) with this ability. Flamingo models include key architectural innovations to: (i) bridge powerful pretrained vision-only and language-only models, (ii) handle sequences of arbitrarily interleaved visual and textual data, and (iii) seamlessly ingest images or videos as inputs. Thanks to their flexibility, Flamingo models can be trained on large-scale multimodal web corpora containing arbitrarily interleaved text and images, which is key to endow them with in-context few-shot learning capabilities. We perform a thorough evaluation of the proposed Flamingo models, exploring and measuring their ability to rapidly adapt to a variety of image and video understanding benchmarks. These include open-ended tasks such as visual question-answering, where the model is prompted with a question which it has to answer, captioning tasks, which evaluate the ability to describe a scene or an event, and close-ended tasks such as multiple choice visual question-answering. For tasks lying anywhere on this spectrum, we demonstrate that a single Flamingo model can achieve a new state of the art for few-shot learning, simply by prompting the model with task-specific examples. On many of these benchmarks, Flamingo actually surpasses the performance of models that are fine-tuned on thousands of times more task-specific data.

show abstract

Section: Risks and Mitigation Strategiesmentioning

confidence: 95%

Flamingo: a Visual Language Model for Few-Shot Learning

Alayrac¹,

Donahue²,

Luc³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Sono poi noti gli innovativi profili di rilevanza del principio di non discriminazione nel secondo scenario: dagli algoritmi discriminatori utilizzati da Amazon nel reclutamento (De Cesco 2018; Huffingtonpost 2018), ai rischi di discriminazione connessi alla diffusione di meccanismi di riconoscimento facciale (Schwemmer et al 2020; Buolamwini e Gebru 2018) e agli effetti di datizzazione e profilazione a essi connessi, che sono costati il posto di lavoro in Google a Timnit Gebru e a Margaret Mitchell 53 . Per poi arrivare, con riguardo al terzo scenario, quello della economia delle piattaforme (Alessi 2019), alla ormai nota vicenda della discriminazione posta in essere dall'algoritmo Frank di Deliveroo (Perulli 2021b) o alle potenziali discriminazioni derivanti da meccanismi reputazionali da parte degli utenti che lasciano aperta la possibilità di una valutazione negativa del servizio reso sulla base di elementi che non riguardano la prestazione in sé, quanto possibili pregiudizi dell'utente per profili razziali o religiosi ecc.…”

Section: Una Possibile Trappola Epistemicaunclassified

Un diritto del lavoro ‘tridimensionale’: valori e tecniche di fronte ai mutamenti dei luoghi di lavoro

Caruso

Zappalà

2022

Studi E Saggi

View full text Add to dashboard Cite

The essay analyses the changes in labour law by examining working places’ modifications induced by the most meaningful driver: technology. In this perspective, the authors point out three macro scenarios. The first relates to industry 4.0 and to the notion of “cognitive enterprise”. This could, or should, act as a magnet model considering its social and organizational positive consequences. The second is the traditional work organization re-considered in the light of new technologies’ impact; it generates a kind of neo digital Fordism, as does the Amazon model in the logistic sector. The third scenario is linked to the rise of the gig economic, with its “workers on tap” (i.e. the platform economy with the algorithm as a boss). These different scenarios, even if characterised by blurred borders, imply the need to think of Labour law differently than in the past. Beyond its traditional protective function, there should be a promotional one, aimed at developing capabilities and positive liberties of the person in the workplace. In the face of this three-fold dimension of the workplaces and of the expansion of Labour law functions, it is also necessary to rethink its techniques and principles. In that sense, the authors propose a different regulatory perspective: no more rigid and uniform rules, as happened in the “short century”, but a regulation made by differentiation, adaptations, extensions, dilutions. With a unifying value: the reference to the individual, to her/his life project, and ultimately to her/his dignity.

show abstract

“…There also exist many research studies exploring gender bias in different types of images. By detecting gender labels of the photographs of U.S. members of Congress and their tweeted images, Schwemmer et al (2020) concluded Google Cloud Vision (GCV) could produce correct and biased labels at the same time because a subset of many possible true labels was selectively reported. Wijnhoven (2021) found a gender bias toward stereotypically female jobs for women but also for men when searching jobs via Google search engine.…”

Section: Related Workmentioning

confidence: 99%

Has CEO Gender Bias Really Been Fixed? Adversarial Attacking and Improving Gender Fairness in Image Search

Feng

Shah

2022

AAAI

View full text Add to dashboard Cite

Gender bias is one of the most common and well-studied demographic biases in information retrieval, and in general in AI systems. After discovering and reporting that gender bias for certain professions could change searchers' worldviews, mainstreaming image search engines, such as Google, quickly took action to correct and fix such a bias. However, given the nature of these systems, viz., being opaque, it is unclear if they addressed unequal gender representation and gender stereotypes in image search results systematically and in a sustainable way. In this paper, we propose adversarial attack queries composed of professions and countries (e.g., 'CEO United States') to investigate whether gender bias is thoroughly mitigated by image search engines. Our experiments on Google, Baidu, Naver, and Yandex Image Search show that the proposed attack can trigger high levels of gender bias in image search results very effectively. To defend against such attacks and mitigate gender bias, we design and implement three novel re-ranking algorithms -- epsilon-greedy algorithm, relevance-aware swapping algorithm, and fairness-greedy algorithm, to re-rank returned images for given image queries. Experiments on both simulated (three typical gender distributions) and real-world datasets demonstrate the proposed algorithms can mitigate gender bias effectively.

show abstract

Diagnosing Gender Bias in Image Recognition Systems

Abstract: This is a brief outline of early work in progress. We will update this document after analyzing results from a crowd sourced validation procedure]

Cited by 10 publications

References 1 publication

Flamingo: a Visual Language Model for Few-Shot Learning

Flamingo: a Visual Language Model for Few-Shot Learning

Un diritto del lavoro ‘tridimensionale’: valori e tecniche di fronte ai mutamenti dei luoghi di lavoro

Has CEO Gender Bias Really Been Fixed? Adversarial Attacking and Improving Gender Fairness in Image Search

Contact Info

Product

Resources

About