Limitations of Pinned AUC for Measuring Unintended Bias

Borkan, Daniel; Dixon, Lucas; Li, John; Sorensen, Jeffrey; Thain, Nithum; Vasserman, Lucy

doi:10.48550/arxiv.1903.02088

Cited by 6 publications

(7 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[5] introduces a threshold agnostic metric for unintended bias, but a follow up work by the same authors, [2], highlights several limitations of this metric. Specifically, the metric is not robust to variations in the class distribution between different identity groups.…”

Section: Related Workmentioning

confidence: 99%

Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification

Borkan

Dixon

Sorensen

et al. 2019

Companion Proceedings of the 2019 World Wide Web Conference

Self Cite

240

276

View full text Add to dashboard Cite

Unintended bias in Machine Learning can manifest as systemic differences in performance for different demographic groups, potentially compounding existing challenges to fairness in society at large. In this paper, we introduce a suite of threshold-agnostic metrics that provide a nuanced view of this unintended bias, by considering the various ways that a classifier's score distribution can vary across designated groups. We also introduce a large new test set of online comments with crowd-sourced annotations for identity references. We use this to show how our metrics can be used to find new and potentially subtle unintended bias in existing public models. CCS CONCEPTS• Computing methodologies → Supervised learning by classification; Model verification and validation. ACM Reference Format:

show abstract

Section: Related Workmentioning

confidence: 99%

Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification

Borkan

Dixon

Sorensen

et al. 2019

Companion Proceedings of the 2019 World Wide Web Conference

Self Cite

240

276

View full text Add to dashboard Cite

show abstract

“…Scenario Description. CivilComments (Borkan et al, 2019a) is a text classification scenario that features examples from the Civil Comments platform, a commenting plugin for independent news sites. As an example, an input for the scenario looks like: "Blame men.…”

Section: B31 Cnn/dmmentioning

confidence: 99%

Holistic Evaluation of Language Models

Liang¹,

Bommasani²,

Lee³

et al. 2022

Preprint

View full text Add to dashboard Cite

Language models (LMs) are becoming the foundation for almost all major language technologies, but their capabilities, limitations, and risks are not well understood. We present Holistic Evaluation of Language Models (HELM) to improve the transparency of language models. First, we taxonomize the vast space of potential scenarios (i.e. use cases) and metrics (i.e. desiderata) that are of interest for LMs. Then we select a broad subset based on coverage and feasibility, noting what's missing or underrepresented (e.g. question answering for neglected English dialects, metrics for trustworthiness). Second, we adopt a multi-metric approach: We measure 7 metrics (accuracy, calibration, robustness, fairness, bias, toxicity, and efficiency) for each of 16 core scenarios to the extent possible (87.5% of the time), ensuring that metrics beyond accuracy don't fall to the wayside, and that trade-offs across models and metrics are clearly exposed. We also perform 7 targeted evaluations, based on 26 targeted scenarios, to more deeply analyze specific aspects (e.g. knowledge, reasoning, memorization/copyright, disinformation). Third, we conduct a large-scale evaluation of 30 prominent language models (spanning open, limited-access, and closed models) on all 42 scenarios, including 21 scenarios that were not previously used in mainstream LM evaluation. Prior to HELM, models on average were evaluated on just 17.9% of the core HELM scenarios, with some prominent models not sharing a single scenario in common. We improve this to 96.0%: now all 30 models have been densely benchmarked on a set of core scenarios and metrics under standardized conditions. Our evaluation surfaces 25 top-level findings concerning the interplay between different scenarios, metrics, and models. For full transparency, we release all raw model prompts and completions publicly 3 for further analysis, as well as a general modular toolkit for easily adding new scenarios, models, metrics, and prompting strategies. 4 We intend for HELM to be a living benchmark for the community, continuously updated with new scenarios, metrics, and models.

show abstract

“…Therefore, Dixon et al (2018) have introduced a measure for unintended bias called Pinned AUC : P inned A rea U nder the C urve to evaluate and compare unintended bias in trained models. However, this measure seems to be inefficient when datasets used as testing are unevenly distributed across different social groups ( Borkan et al, 2019 ) .…”

Section: Background and Related Workmentioning

confidence: 99%

Syntax and prejudice: ethically-charged biases of a syntax-based hate speech recognizer unveiled

Mastromattei

Ranaldi

Fallucchi

et al. 2022

PeerJ Computer Science

View full text Add to dashboard Cite

Hate speech recognizers (HSRs) can be the panacea for containing hate in social media or can result in the biggest form of prejudice-based censorship hindering people to express their true selves. In this paper, we hypothesized how massive use of syntax can reduce the prejudice effect in HSRs. To explore this hypothesis, we propose Unintended-bias Visualizer based on Kermit modeling (KERM-HATE): a syntax-based HSR, which is endowed with syntax heat parse trees used as a post-hoc explanation of classifications. KERM-HATE significantly outperforms BERT-based, RoBERTa-based and XLNet-based HSR on standard datasets. Surprisingly this result is not sufficient. In fact, the post-hoc analysis on novel datasets on recent divisive topics shows that even KERM-HATE carries the prejudice distilled from the initial corpus. Therefore, although tests on standard datasets may show higher performance, syntax alone cannot drive the “attention” of HSRs to ethically-unbiased features.

show abstract

Limitations of Pinned AUC for Measuring Unintended Bias

Cited by 6 publications

References 0 publications

Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification

Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification

Holistic Evaluation of Language Models

Syntax and prejudice: ethically-charged biases of a syntax-based hate speech recognizer unveiled

Contact Info

Product

Resources

About