Human Uncertainty in Concept-Based AI Systems

Collins, Katherine Maeve; Barker, Matthew; Espinosa Zarlenga, Mateo; Raman, Naveen; Bhatt, Umang; Jamnik, Mateja; Sucholutsky, Ilia; Weller, Adrian; Dvijotham, Krishnamurthy

doi:10.1145/3600211.3604692

Cited by 2 publications

(2 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We encourage further investigation into discrepancies between human and algorithm concept preferences -and potential for complementarity (Steyvers et al 2022) -through CUB-Sel and related datasets such as CUB-S (Collins et al 2023). In addition, smarter intervention methods, such as CooP (Chauhan et al 2023), could be used in conjunction with SCOMs to further increase the efficacy of interventions.…”

Section: Future Workmentioning

confidence: 99%

“…SCOM reduces the size of the concept set used for prediction, increasing the impact of each intervention, while still allowing intervention procedures to select the best concept to intervene on. While in practice, stakeholders may prefer to provide soft labels if they are unsure, (Collins et al 2023) show that existing models perform poorly with uncertain interventions.…”

Section: Comparison To Prior Workmentioning

confidence: 99%

See 1 more Smart Citation

Selective Concept Models: Permitting Stakeholder Customisation at Test-Time

Barker,

Collins,

Dvijotham

et al. 2023

HCOMP

View full text Add to dashboard Cite

Concept-based models perform prediction using a set of concepts that are interpretable to stakeholders. However, such models often involve a fixed, large number of concepts, which may place a substantial cognitive load on stakeholders. We propose Selective COncept Models (SCOMs) which make predictions using only a subset of concepts and can be customised by stakeholders at test-time according to their preferences. We show that SCOMs only require a fraction of the total concepts to achieve optimal accuracy on multiple real-world datasets. Further, we collect and release a new dataset, CUB-Sel, consisting of human concept set selections for 900 bird images from the popular CUB dataset. Using CUB-Sel, we show that humans have unique individual preferences for the choice of concepts they prefer to reason about, and struggle to identify the most theoretically informative concepts. The customisation and concept selection provided by SCOM improves the efficiency of interpretation and intervention for stakeholders.

show abstract

Section: Future Workmentioning

confidence: 99%

Section: Comparison To Prior Workmentioning

confidence: 99%

Selective Concept Models: Permitting Stakeholder Customisation at Test-Time

Barker,

Collins,

Dvijotham

et al. 2023

HCOMP

View full text Add to dashboard Cite

show abstract

Addressing the Binning Problem in Calibration Assessment through Scalar Annotations

Jiang,

Liu,

Durme

2024

Transactions of the Association for Computational Linguistics

View full text Add to dashboard Cite

Computational linguistics models commonly target the prediction of discrete—categorical—labels. When assessing how well-calibrated these model predictions are, popular evaluation schemes require practitioners to manually determine a binning scheme: grouping labels into bins to approximate true label posterior. The problem is that these metrics are sensitive to binning decisions. We consider two solutions to the binning problem that apply at the stage of data annotation: collecting either distributed (redundant) labels or direct scalar value assignment. In this paper, we show that although both approaches address the binning problem by evaluating instance-level calibration, direct scalar assignment is significantly more cost-effective. We provide theoretical analysis and empirical evidence to support our proposal for dataset creators to adopt scalar annotation protocols to enable a higher-quality assessment of model calibration.

show abstract

Human Uncertainty in Concept-Based AI Systems

Cited by 2 publications

References 31 publications

Selective Concept Models: Permitting Stakeholder Customisation at Test-Time

Selective Concept Models: Permitting Stakeholder Customisation at Test-Time

Addressing the Binning Problem in Calibration Assessment through Scalar Annotations

Contact Info

Product

Resources

About