Fair-VQA: Fairness-Aware Visual Question Answering Through Sensitive Attribute Prediction

Park, Sungho; Hwang, Sunhee; Hong, Jongkwang; Byun, Hyeran

doi:10.1109/access.2020.3041503

Cited by 7 publications

(2 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Beyond the supervised learning, FairFaceGAN (Hwang et al, 2020) is proposed to prevent undesired sensitive feature translation during image editing. Similar ideas have also been successfully applied to visual question answering (Park et al, 2020).…”

Section: Machine Learning Fairnessmentioning

confidence: 98%

How Robust is Your Fairness? Evaluating and Sustaining Fairness under Unseen Distribution Shifts

Wang,

Hong,

Zhou

et al. 2022

Preprint

View full text Add to dashboard Cite

Increasing concerns have been raised on deep learning fairness in recent years. Existing fairnessaware machine learning methods mainly focus on the fairness of in-distribution data. However, in real-world applications, it is common to have distribution shift between the training and test data. In this paper, we first show that the fairness achieved by existing methods can be easily broken by slight distribution shifts. To solve this problem, we propose a novel fairness learning method termed CUrvature MAtching (CUMA), which can achieve robust fairness generalizable to unseen domains with unknown distributional shifts. Specifically, CUMA enforces the model to have similar generalization ability on the majority and minority groups, by matching the loss curvature distributions of the two groups. We evaluate our method on three popular fairness datasets. Compared with existing methods, CUMA achieves superior fairness under unseen distribution shifts, without sacrificing either the overall accuracy or the in-distribution fairness.

show abstract

Section: Machine Learning Fairnessmentioning

confidence: 98%

How Robust is Your Fairness? Evaluating and Sustaining Fairness under Unseen Distribution Shifts

Wang,

Hong,

Zhou

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…It enhances accuracy by utilizing a CNN that predicts bounding boxes for objects in an image. https://www.indjst.org/ (54) CPDR (55) , MulFA/UFSCAN (56) , Bilinear Graph (57) , AttReg (58) , AMAM (16) , Scene-text using PHOC (59) , MGRF (60) , Bottom-Up and Top-Down (61) , DCAMN (39) , Skill Concept (62) , PGM (63) , SR-OCE (64) , RAMEN (65) , CSST (66) , Coarse-to-Fine (67) , GMA (68) , BLOCK (69) , CapsAtt (32,40) , Re-attention (70) , CRN (71) , CAT (11) , shortcut (72) , DAQC (15) , MGFAN (73) , MMMH (19) , MSG (74) , Fair-VQA (75) , Attention map (5) , SAVQA (76) , MGAVQA (77) , MuKEA (78) , ACVRM (79) , QD-GFN (23) , Swap-Mix (80) , CVA (17) , HGNMN (26) , SUPER (37) , Uncertainty based (81) , CLG (82) , WSQG (83) , VLR (84) , LXMERT (85) , SceneGATE (86)…”

Section: Visual Feature Extraction Techniquesmentioning

confidence: 99%

A Review of Recent Advances in Visual Question Answering: Capsule Networks and Vision Transformers in Focus

Prakash,

Devananda

2024

IJST

View full text Add to dashboard Cite

Objectives: Multimodal deep learning, incorporating images, text, videos, speech, and acoustic signals, has grown significantly. This article aims to explore the untapped possibilities of multimodal deep learning in Visual Question Answering (VQA) and address a research gap in the development of effective techniques for comprehensive image feature extraction. Methods: This article provides a comprehensive overview of VQA and the associated challenges. It emphasizes the need for an extensive representation of images in VQA and pinpoints the specific research gap pertaining to image feature extraction and highlights the fundamental concepts of VQA, the challenges faced, different approaches and applications used for VQA tasks. A substantial portion of this review is devoted to investigating recent advancements in image feature extraction techniques. Findings: Most existing VQA research predominantly emphasizes the accurate matching of answers to given questions, often overlooking the necessity for a comprehensive representation of images. These models primarily rely on question content analysis while underemphasizing image understanding or sometimes neglect image examination entirely. There is also a tendency in multimodal systems to neglect or overemphasize one modality, notably the visual one, which challenges genuine multimodal integration. This article reveals that there is limited benchmarking for image feature extraction techniques. Evaluating the quality of extracted image features is crucial for VQA tasks. Novelty: While many VQA studies have primarily concentrated on the accuracy of answers to questions, this review emphasizes the importance of comprehensive image representation. The paper explores recent advances in Capsules Networks (CapsNets) and Vision Transformers (ViTs) as alternatives to traditional Convolutional Neural Networks (CNNs), for development of more effective image feature extraction techniques which can help to address the limitations of existing VQA models that focus primarily on question content analysis. https://www.indjst.org/

show abstract

Design and development of counting-based visual question answering model using heuristic-based feature selection with deep learning

Welde

Liao

2023

Artif Intell Rev

View full text Add to dashboard Cite

Fair-VQA: Fairness-Aware Visual Question Answering Through Sensitive Attribute Prediction

Cited by 7 publications

References 26 publications

How Robust is Your Fairness? Evaluating and Sustaining Fairness under Unseen Distribution Shifts

How Robust is Your Fairness? Evaluating and Sustaining Fairness under Unseen Distribution Shifts

A Review of Recent Advances in Visual Question Answering: Capsule Networks and Vision Transformers in Focus

Design and development of counting-based visual question answering model using heuristic-based feature selection with deep learning

Contact Info

Product

Resources

About