Visual Psychophysics for Making Face Recognition Algorithms More Explainable

RichardWebster, Brandon; Kwon, So Yon; Clarizio, Christopher; Anthony, Samuel; Scheirer, Walter J.

doi:10.1007/978-3-030-01267-0_16

Cited by 33 publications

(32 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In vision science, psychophysical experiments investigate the relationship between the intensity of a physical stimulus and human perception, by systematically varying the properties of the stimulus along one or more physical dimensions [72]. They are widely used in computer vision research to evaluate an algorithm's behaviour by measuring the exemplar by exemplar diculty and modeling the algorithm's pattern of errors over di erent levels of object visibility and saliency, making the algorithm's classi cation inference more explainable (see examples in [19,27,28,34,63]). In addition, we assessed how the drug type a ected the algorithm's classi cation con dence score.…”

Section: Evaluation Designmentioning

confidence: 99%

An explainable algorithm for detecting drug-induced QT-prolongation at risk of torsades de pointes (TdP) regardless of heart rate and T-wave morphology

Alahmadi

Davies

Royle³

et al. 2021

Computers in Biology and Medicine

View full text Add to dashboard Cite

Section: Evaluation Designmentioning

confidence: 99%

An explainable algorithm for detecting drug-induced QT-prolongation at risk of torsades de pointes (TdP) regardless of heart rate and T-wave morphology

Alahmadi

Davies

Royle³

et al. 2021

Computers in Biology and Medicine

View full text Add to dashboard Cite

“…Martínez-Plumed et al (2019) used IRT to analyze the performance of machine learning classifiers in a supervised learning task. IRT has also been used to evaluate machine translation systems (Otani et al, 2016) and speech synthesizers (Oliveira et al, 2020), and also in computer vision (RichardWebster et al, 2018).…”

Section: Related Workmentioning

confidence: 99%

Can Transformer Language Models Predict Psychometric Properties?

Laverghetta¹,

Nighojkar²,

Mirzakhalov³

et al. 2021

Proceedings of *SEM 2021: The Tenth Joint Conference on Lexical and Computational Semantics

View full text Add to dashboard Cite

Transformer-based language models (LMs) continue to advance state-of-the-art performance on NLP benchmark tasks, including tasks designed to mimic human-inspired "commonsense" competencies. To better understand the degree to which LMs can be said to have certain linguistic reasoning skills, researchers are beginning to adapt the tools and concepts of the field of psychometrics. But to what extent can the benefits flow in the other direction? I.e., can LMs be of use in predicting what the psychometric properties of test items will be when those items are given to human participants? We gather responses from numerous human participants and LMs (transformerand non-transformer-based) on a broad diagnostic test of linguistic competencies. We then use the responses to calculate standard psychometric properties of the items in the diagnostic test, using the human responses and the LM responses separately. We then determine how well these two sets of predictions match. We find cases in which transformerbased LMs predict psychometric properties consistently well in certain categories but consistently poorly in others, thus providing new insights into fundamental similarities and differences between human and LM reasoning. 1

show abstract

“…O'Toole et al [40] demonstrated that machines were never less accurate than humans on face images of various quality. RichardWebster et al [44] showed that observing human face recognition behavior in certain contexts can be used to retroactively explain why a face matcher succeeds or fails, leading to better model explainability. In the realm of biometrics, human saliency was found complementary to algorithm saliency and thus beneficial to combine them [38,49].…”

Section: Use Of Human Perception To Understand and Improvementioning

confidence: 99%

CYBORG: Blending Human Saliency Into the Loss Improves Deep Learning

Boyd¹,

Tinsley²,

Bowyer³

et al. 2021

Preprint

View full text Add to dashboard Cite

Can deep learning models achieve greater generalization if their training is guided by reference to human perceptual abilities? And how can we implement this in a practical manner? This paper proposes a first-ever training strategy to ConveY Brain Oversight to Raise Generalization (CY-BORG). This new training approach incorporates humanannotated saliency maps into a CYBORG loss function that guides the model towards learning features from image regions that humans find salient when solving a given visual task. The Class Activation Mapping (CAM) mechanism is used to probe the model's current saliency in each training batch, juxtapose model saliency with human saliency, and penalize the model for large differences. Results on the task of synthetic face detection show that the CYBORG loss leads to significant improvement in performance on unseen samples consisting of face images generated from six Generative Adversarial Networks (GANs) across multiple classification network architectures. We also show that scaling to even seven times as much training data with standard loss cannot beat the accuracy of CYBORG loss. As a side effect, we observed that the addition of explicit region annotation to the task of synthetic face detection increased human classification performance. This work opens a new area of research on how to incorporate human visual saliency into loss functions. All data, code and pre-trained models used in this work are offered with this paper.

show abstract

Visual Psychophysics for Making Face Recognition Algorithms More Explainable

Cited by 33 publications

References 71 publications

An explainable algorithm for detecting drug-induced QT-prolongation at risk of torsades de pointes (TdP) regardless of heart rate and T-wave morphology

An explainable algorithm for detecting drug-induced QT-prolongation at risk of torsades de pointes (TdP) regardless of heart rate and T-wave morphology

Can Transformer Language Models Predict Psychometric Properties?

CYBORG: Blending Human Saliency Into the Loss Improves Deep Learning

Contact Info

Product

Resources

About