The Values Encoded in Machine Learning Research

Birhane, Abeba; Kalluri, Pratyusha; Card, Dallas; Agnew, William S.; Dotan, Ravit; Bao, Michelle

doi:10.48550/arxiv.2106.15590

Cited by 25 publications

(51 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The culture in machine learning is such that ideas that promise improvements in training speed, modelsize or top-k accuracy improvements are rapidly embraced while ideas and revelations pertaining to unethical aspects of datasets are either ignored or take a long time to lead to changes [62]. For example, the ImageNet dataset was released in 2009 [63] but the course-corrections regarding the vast number of non-imageable classes [12] and loss of privacy [14] were undertaken only in the 2019-21 period which is more than a decade after its release.…”

Section: Asymmetry Of 'Advances': Model Advances V/s Dataset Advancesmentioning

confidence: 99%

Multimodal datasets: misogyny, pornography, and malignant stereotypes

Birhane,

Prabhu,

Kahembwe

2021

Preprint

Self Cite

View full text Add to dashboard Cite

We have now entered the era of trillion parameter machine learning models trained on billion-sized datasets scraped from the internet. The rise of these gargantuan datasets has given rise to formidable bodies of critical work that has called for caution while generating these large datasets. These address concerns surrounding the dubious curation practices used to generate these datasets, the sordid quality of alt-text data available on the world wide web, the problematic content of the CommonCrawl dataset often used as a source for training large language models, and the entrenched biases in large-scale visio-linguistic models (such as OpenAI's CLIP model) trained on opaque datasets (WebImageText). In the backdrop of these specific calls of caution, we examine the recently released LAION-400M dataset, which is a CLIP-filtered dataset of Image-Alt-text pairs parsed from the Common-Crawl dataset. We found that the dataset contains, troublesome and explicit images and text pairs of rape, pornography, malign stereotypes, racist and ethnic slurs, and other extremely problematic content. We outline numerous implications, concerns and downstream harms regarding the current state of large scale datasets while raising open questions for various stakeholders including the AI community, regulators, policy makers and data subjects. Warning: This paper contains NSFW content that some readers may find disturbing, distressing, and/or offensive.

show abstract

Section: Asymmetry Of 'Advances': Model Advances V/s Dataset Advancesmentioning

confidence: 99%

Multimodal datasets: misogyny, pornography, and malignant stereotypes

Birhane,

Prabhu,

Kahembwe

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The three are commonly used constructs for usability measures [46]. This reflects a deep value of the field [17], which prioritizes optimizing decision outcomes rather than the experience of human decision makers. That being said, we acknowledge that not all decision tasks require high efficiency (also the efficiency of AI alone is trivially better than human-AI teams).…”

Section: Summary and Takeawaysmentioning

confidence: 99%

Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies

Lai¹,

Chen²,

Liao³

et al. 2021

Preprint

View full text Add to dashboard Cite

As AI systems demonstrate increasingly strong predictive performance, their adoption has grown in numerous domains. However, in high-stakes domains such as criminal justice and healthcare, full automation is often not desirable due to safety, ethical, and legal concerns, yet fully manual approaches can be inaccurate and time consuming. As a result, there is growing interest in the research community to augment human decision making with AI assistance. Besides developing AI technologies for this purpose, the emerging field of human-AI decision making must embrace empirical approaches to form a foundational understanding of how humans interact and work with AI to make decisions. To invite and help structure research efforts towards a science of understanding and improving human-AI decision making, we survey recent literature of empirical human-subject studies on this topic. We summarize the study design choices made in over 100 papers in three important aspects: (1) decision tasks, (2) AI models and AI assistance elements, and(3) evaluation metrics. For each aspect, we summarize current trends, discuss gaps in current practices of the field, and make a list of recommendations for future research. Our survey highlights the need to develop common frameworks to account for the design and research spaces of human-AI decision making, so that researchers can make rigorous choices in study design, and the research community can build on each other's work and produce generalizable scientific knowledge. We also hope this survey will serve as a bridge for HCI and AI communities to work together to mutually shape the empirical science and computational technologies for human-AI decision making.

show abstract

“…It is widely known that ML algorithms and tools, like any other set of computational modeling schemes, are not automatically trustworthy. Moreover, significant peril accompanies their use in contexts that require unbiased or quantifiably precise results: these circumstances are well known to occur in both science and society (e.g., Griffiths et al 2021;Ćiprijanović et al 2021;Birhane & Cummins 2019;Birhane et al 2021). It is imperative that ML practitioners proceed with caution.…”

Section: Scrutinize All Resultsmentioning

confidence: 99%

Building Trustworthy Machine Learning Models for Astronomy

Ntampaka

Nord

2021

Preprint

View full text Add to dashboard Cite

Astronomy is entering an era of data-driven discovery, due in part to modern machine learning (ML) techniques enabling powerful new ways to interpret observations. This shift in our scientific approach requires us to consider whether we can trust the black box. Here, we overview methods for an often-overlooked step in the development of ML models: building community trust in the algorithms. Trust is an essential ingredient not just for creating more robust data analysis techniques, but also for building confidence within the astronomy community to embrace machine learning methods and results.

show abstract

The Values Encoded in Machine Learning Research

Cited by 25 publications

References 0 publications

Multimodal datasets: misogyny, pornography, and malignant stereotypes

Multimodal datasets: misogyny, pornography, and malignant stereotypes

Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies

Building Trustworthy Machine Learning Models for Astronomy

Contact Info

Product

Resources

About