Collaborative Human Machine Attention Module for Character Recognition

Ralekar, Chetan; Gandhi, Tapan Kumar; Chaudhury, Santanu

doi:10.1109/icpr48806.2021.9413229

Cited by 1 publication

(2 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nowadays, DNNs are approaching human-level performance in recognition, and various other computer vision tasks through network engineering wherein the factors such as depth, width, and cardinality of the model are varied [49]. In addition to network engineering, attention mechanisms are also incorporated in the models [49]- [53]. In most of the cases, these methods requires additional parameters and hyper-parameters to be tuned for the best possible performance.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Understanding Character Recognition using Visual Explanations Derived from the Human Visual System and Deep Networks

Ralekar,

Choudhary,

Gandhi

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Human observers engage in selective information uptake when classifying visual patterns. The same is true of deep neural networks, which currently constitute the best performing artificial vision systems. Our goal is to examine the congruence, or lack thereof, in the information-gathering strategies of the two systems. We have operationalized our investigation as a character recognition task. We have used eye-tracking to assay the spatial distribution of information hotspots for humans via fixation maps and an activation mapping technique for obtaining analogous distributions for deep networks through visualization maps. Qualitative comparison between visualization maps and fixation maps reveals an interesting correlate of congruence. The deep learning model considered similar regions in character, which humans have fixated in the case of correctly classified characters. On the other hand, when the focused regions are different for humans and deep nets, the characters are typically misclassified by the latter. Hence, we propose to use the visual fixation maps obtained from the eye-tracking experiment as a supervisory input to align the model's focus on relevant character regions. We find that such supervision improves the model's performance significantly and does not require any additional parameters. This approach has the potential to find applications in diverse domains such as medical analysis and surveillance in which explainability helps to determine system fidelity.

show abstract

Section: Discussionmentioning

confidence: 99%

“…The visual attention derived from eye-fixations has been used in some studies [52], [53]. However, these models hardly consider the explanability.…”

Section: Discussionmentioning

confidence: 99%

Understanding Character Recognition using Visual Explanations Derived from the Human Visual System and Deep Networks

Ralekar,

Choudhary,

Gandhi

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Collaborative Human Machine Attention Module for Character Recognition

Cited by 1 publication

References 24 publications

Understanding Character Recognition using Visual Explanations Derived from the Human Visual System and Deep Networks

Understanding Character Recognition using Visual Explanations Derived from the Human Visual System and Deep Networks

Contact Info

Product

Resources

About