Intelligent Identification of Ornamental Devanagari Characters Inspired by Visual Fixations

Ralekar, Chetan; Choudhary, Shubham; Gandhi, Tapan Kumar; Chaudhury, Santanu

doi:10.1109/icdarw.2019.40074

Cited by 4 publications

(2 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The visual attention derived from eye-fixations has been used in some studies [52], [53]. However, these models hardly consider the explanability.…”

Section: Discussionmentioning

confidence: 99%

Understanding Character Recognition using Visual Explanations Derived from the Human Visual System and Deep Networks

Ralekar,

Choudhary,

Gandhi

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Human observers engage in selective information uptake when classifying visual patterns. The same is true of deep neural networks, which currently constitute the best performing artificial vision systems. Our goal is to examine the congruence, or lack thereof, in the information-gathering strategies of the two systems. We have operationalized our investigation as a character recognition task. We have used eye-tracking to assay the spatial distribution of information hotspots for humans via fixation maps and an activation mapping technique for obtaining analogous distributions for deep networks through visualization maps. Qualitative comparison between visualization maps and fixation maps reveals an interesting correlate of congruence. The deep learning model considered similar regions in character, which humans have fixated in the case of correctly classified characters. On the other hand, when the focused regions are different for humans and deep nets, the characters are typically misclassified by the latter. Hence, we propose to use the visual fixation maps obtained from the eye-tracking experiment as a supervisory input to align the model's focus on relevant character regions. We find that such supervision improves the model's performance significantly and does not require any additional parameters. This approach has the potential to find applications in diverse domains such as medical analysis and surveillance in which explainability helps to determine system fidelity.

show abstract

“…The visual attention derived from eye-fixations has been used in some studies [52], [53]. However, these models hardly consider the explanability.…”

Section: Discussionmentioning

confidence: 99%

Understanding Character Recognition using Visual Explanations Derived from the Human Visual System and Deep Networks

Ralekar,

Choudhary,

Gandhi

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Grad-CAM methods are currently the most widely used methods for this type of question, while studies using gradientbased CAM methods combined with eye-movement analysis are still relatively rare, and we believe this is an area worth developing, and some researchers are currently trying and following up on this, for example, in this study (Alarifi et al, 2019) comparing eye-tracking experiments with activation maps for age estimation, and in another recent study (Ralekar et al, 2021), the authors also compared eye-tracking data with neural network attention regions for character recognition, combining eye-tracking analysis with CAM methods in this paper (Langlois et al, 2021). It is important to note that there are still few studies that combine the two.…”

Section: Attention Area Analysis Of Neural Network and Humanmentioning

confidence: 94%

Disrupted visual input unveils the computational details of artificial neural networks for face perception

Ying

2022

Front. Comput. Neurosci.

View full text Add to dashboard Cite

BackgroundConvolutional Neural Network (DCNN), with its great performance, has attracted attention of researchers from many disciplines. The studies of the DCNN and that of biological neural systems have inspired each other reciprocally. The brain-inspired neural networks not only achieve great performance but also serve as a computational model of biological neural systems.MethodsHere in this study, we trained and tested several typical DCNNs (AlexNet, VGG11, VGG13, VGG16, DenseNet, MobileNet, and EfficientNet) with a face ethnicity categorization task for experiment 1, and an emotion categorization task for experiment 2. We measured the performance of DCNNs by testing them with original and lossy visual inputs (various kinds of image occlusion) and compared their performance with human participants. Moreover, the class activation map (CAM) method allowed us to visualize the foci of the “attention” of these DCNNs.ResultsThe results suggested that the VGG13 performed the best: Its performance closely resembled human participants in terms of psychophysics measurements, it utilized similar areas of visual inputs as humans, and it had the most consistent performance with inputs having various kinds of impairments.DiscussionIn general, we examined the processing mechanism of DCNNs using a new paradigm and found that VGG13 might be the most human-like DCNN in this task. This study also highlighted a possible paradigm to study and develop DCNNs using human perception as a benchmark.

show abstract

Collaborative Human Machine Attention Module for Character Recognition

Ralekar

Gandhi

Chaudhury

2021

2020 25th International Conference on Pattern Recognition (ICPR)

View full text Add to dashboard Cite

Intelligent Identification of Ornamental Devanagari Characters Inspired by Visual Fixations

Cited by 4 publications

References 15 publications

Understanding Character Recognition using Visual Explanations Derived from the Human Visual System and Deep Networks

Understanding Character Recognition using Visual Explanations Derived from the Human Visual System and Deep Networks

Disrupted visual input unveils the computational details of artificial neural networks for face perception

Collaborative Human Machine Attention Module for Character Recognition

Contact Info

Product

Resources

About