Dataset bias: A case study for visual question answering

Das, Anubrata; Anjum, Samreen; Gurari, Danna

doi:10.1002/pra2.7

Cited by 9 publications

(15 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, bias has been analyzed for commercial video clips [36], automated facial analysis technologies [48], algorithms that predict whether a convicted criminal will re-offend [54], loan lending algorithms [45], and LabInTheWild studies [8]. More similar to our work, it has been shown that it is possible, for a given visual question, to identify which dataset it belongs to from a known collection of VQA datasets [16,19]. Similar to prior work, our findings expose biases in datasets, and so underscore important next 149:4 Xiaoyu Zeng et al…”

Section: Related Workmentioning

confidence: 85%

“…1 Altogether, our work offers valuable insights to multiple stakeholders -creators of computer vision datasets, designers of VQA systems, and users of VQA systems. It is broadly-known that computer vision datasets can embed flawed assumptions that lead to perpetuating and amplifying biases in technology, including for VQA [16,19]. Yet, the types of biases can be unknown to those developing the AI.…”

Section: :3mentioning

confidence: 99%

See 1 more Smart Citation

Vision Skills Needed to Answer Visual Questions

Zeng

Wang

Chiu

et al. 2020

Proc. ACM Hum.-Comput. Interact.

Self Cite

View full text Add to dashboard Cite

The task of answering questions about images has garnered attention as a practical service for assisting populations with visual impairments as well as a visual Turing test for the artificial intelligence community. Our first aim is to identify the common vision skills needed for both scenarios. To do so, we analyze the need for four vision skills-object recognition, text recognition, color recognition, and counting-on over 27,000 visual questions from two datasets representing both scenarios. We next quantify the difficulty of these skills for both humans and computers on both datasets. Finally, we propose a novel task of predicting what vision skills are needed to answer a question about an image. Our results reveal (mis)matches between aims of real users of such services and the focus of the AI community. We conclude with a discussion about future directions for addressing the visual question answering task.

show abstract

Section: Related Workmentioning

confidence: 85%

Section: :3mentioning

confidence: 99%

Vision Skills Needed to Answer Visual Questions

Zeng

Wang

Chiu

et al. 2020

Proc. ACM Hum.-Comput. Interact.

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the study of [32] The detection of bias in developed models enables the recognition of whether questions are answered by a human with normal vision, a blind person, or a robot. It does so by utilizing visual question answering (VQA) datasets cataloged by individuals with normal vision, blind people, and a robot.…”

Section: Mitigation Techniques and Modelsmentioning

confidence: 99%

Bias and unfairness in machine learning models: a systematic literature review

Pagano¹,

Loureiro²,

Araujo³

et al. 2022

Preprint

View full text Add to dashboard Cite

One of the difficulties of artificial intelligence is to ensure that model decisions are fair and free of bias. In research, datasets, metrics, techniques, and tools are applied to detect and mitigate algorithmic unfairness and bias. This study aims to examine existing knowledge on bias and unfairness in Machine Learning models, identifying mitigation methods, fairness metrics, and supporting tools. A Systematic Literature Review found 40 eligible articles published between 2017 and 2022 in the Scopus, IEEE Xplore, Web of Science, and Google Scholar knowledge bases. The results show numerous bias and unfairness detection and mitigation approaches for ML technologies, with clearly defined metrics in the literature, and varied metrics can be highlighted. We recommend further research to define the techniques and metrics that should be employed in each case to standardize and ensure the impartiality of the machine learning model, thus, allowing the most appropriate metric to detect bias and unfairness in a given context.

show abstract

“…As a result of annotation artifacts, existing NLP datasets contain shallow patterns that correlate with target labels (Gururangan et al, 2018;McCoy et al, 2019;Schuster et al, 2019a;Le Bras et al, 2020;Jia and Liang, 2017;Das et al, 2019). Models tend to exploit these shallow patterns-which we refer to as biases in this paper-instead of learning general knowledge about solving the target task.…”

Section: Introductionmentioning

confidence: 99%

Improving QA Generalization by Concurrent Modeling of Multiple Biases

Wu¹,

Moosavi

Rücklé

et al. 2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

Existing NLP datasets contain various biases that models can easily exploit to achieve high performances on the corresponding evaluation sets. However, focusing on dataset-specific biases limits their ability to learn more generalizable knowledge about the task from more general data patterns. In this paper, we investigate the impact of debiasing methods for improving generalization and propose a general framework for improving the performance on both in-domain and out-of-domain datasets by concurrent modeling of multiple biases in the training data. Our framework weights each example based on the biases it contains and the strength of those biases in the training data. It then uses these weights in the training objective so that the model relies less on examples with high bias weights. We extensively evaluate our framework on extractive question answering with training data from various domains with multiple biases of different strengths. We perform the evaluations in two different settings, in which the model is trained on a single domain or multiple domains simultaneously, and show its effectiveness in both settings compared to state-of-the-art debiasing methods. 1

show abstract

Dataset bias: A case study for visual question answering

Cited by 9 publications

References 35 publications

Vision Skills Needed to Answer Visual Questions

Vision Skills Needed to Answer Visual Questions

Bias and unfairness in machine learning models: a systematic literature review

Improving QA Generalization by Concurrent Modeling of Multiple Biases

Contact Info

Product

Resources

About