Re-imagining Algorithmic Fairness in India and Beyond

Sambasivan, Nithya; Arnesen, Erin; Hutchinson, Ben; Doshi, Tulsee; Prabhakaran, Vinodkumar

doi:10.48550/arxiv.2101.09995

Cited by 3 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our safety objectives aim to reduce the number of responses biased against specific subgroups of people, but such biases can be hard to detect since they manifest in a wide variety of subtle ways. For example, the axes of marginalization differ greatly across geo-cultural contexts, and how they manifest in pre-trained language models is an under-studied area [101].…”

Section: Examining Biasmentioning

confidence: 99%

“…While these efforts are important, it is critical to also consider the downstream applications and the socio-technical ecosystems where they will be deployed when measuring the impact of these efforts in mitigating harm. For example, bias mitigations in certain contexts might have counter-intuitive impacts in other geocultural contexts [101].…”

Section: Examining Biasmentioning

confidence: 99%

See 1 more Smart Citation

LaMDA: Language Models for Dialog Applications

Thoppilan¹,

Freitas²,

Hall³

et al. 2022

Preprint

Self Cite

186

270

View full text Add to dashboard Cite

We present LaMDA: Language Models for Dialog Applications. LaMDA is a family of Transformerbased neural language models specialized for dialog, which have up to 137B parameters and are pre-trained on 1.56T words of public dialog data and web text. While model scaling alone can improve quality, it shows less improvements on safety and factual grounding. We demonstrate that fine-tuning with annotated data and enabling the model to consult external knowledge sources can lead to significant improvements towards the two key challenges of safety and factual grounding. The first challenge, safety, involves ensuring that the model's responses are consistent with a set of human values, such as preventing harmful suggestions and unfair bias. We quantify safety using a metric based on an illustrative set of human values, and we find that filtering candidate responses using a LaMDA classifier fine-tuned with a small amount of crowdworker-annotated data offers a promising approach to improving model safety. The second challenge, factual grounding, involves enabling the model to consult external knowledge sources, such as an information retrieval system, a language translator, and a calculator. We quantify factuality using a groundedness metric, and we find that our approach enables the model to generate responses grounded in known sources, rather than responses that merely sound plausible. Finally, we explore the use of LaMDA in the domains of education and content recommendations, and analyze their helpfulness and role consistency. * Work done while at Google.

show abstract

Section: Examining Biasmentioning

confidence: 99%

Section: Examining Biasmentioning

confidence: 99%

LaMDA: Language Models for Dialog Applications

Thoppilan¹,

Freitas²,

Hall³

et al. 2022

Preprint

Self Cite

186

270

View full text Add to dashboard Cite

show abstract

“…As discussed in [52], considering human elicited metrics might be an avenue forward. Similarly, 'fairness' might change depending on the context [59] and different metrics might be relevant in different environments.…”

Section: 34mentioning

confidence: 99%

Maintaining fairness across distribution shift: do we have viable solutions for real-world applications?

Schrouff¹,

Harris²,

Koyejo³

et al. 2022

Preprint

View full text Add to dashboard Cite

Fairness and robustness are often considered as orthogonal dimensions when evaluating machine learning models. However, recent work has revealed interactions between fairness and robustness, showing that fairness properties are not necessarily maintained under distribution shift. In healthcare settings, this can result in e.g. a model that performs fairly according to a selected metric in "hospital A" showing unfairness when deployed in "hospital B". While a nascent field has emerged to develop provable fair and robust models, it typically relies on strong assumptions about the shift, limiting its impact for real-world applications. In this work, we explore the settings in which recently proposed mitigation strategies are applicable by referring to a causal framing. Using examples of predictive models in dermatology and electronic health records, we show that real-world applications are complex and often invalidate the assumptions of such methods. Our work hence highlights technical, practical, and engineering gaps that prevent the development of robustly fair machine learning models for real-world applications. Finally, we discuss potential remedies at each step of the machine learning pipeline.

show abstract

“…The discriminated sub-groups, their values, and underlying social constructs may also differ across communities: e.g. both in India and US there is discrimination based on skin tone, but in the US context it stands for race, and in India it is a proxy for ethnicity, caste and class (Sambasivan et al, 2021a).…”

Section: 26mentioning

confidence: 99%

Changing the World by Changing the Data

Rogers¹

2021

Preprint

View full text Add to dashboard Cite

NLP community is currently investing a lot more research and resources into development of deep learning models than training data. While we have made a lot of progress, it is now clear that our models learn all kinds of spurious patterns, social biases, and annotation artifacts. Algorithmic solutions have so far had limited success. An alternative that is being actively discussed is more careful design of datasets so as to deliver specific signals. This position paper maps out the arguments for and against data curation, and argues that fundamentally the point is moot: curation already is and will be happening, and it is changing the world. The question is only how much thought we want to invest into that process.

show abstract

Re-imagining Algorithmic Fairness in India and Beyond

Cited by 3 publications

References 0 publications

LaMDA: Language Models for Dialog Applications

LaMDA: Language Models for Dialog Applications

Maintaining fairness across distribution shift: do we have viable solutions for real-world applications?

Changing the World by Changing the Data

Contact Info

Product

Resources

About