Study on Representation Invariances of CNNs and Human Visual Information Processing Based on Data Augmentation

Abstract

Representation invariance plays a significant role in the performance of deep convolutional neural networks (CNNs) and human visual information processing in various complicated image-based tasks. However, there has been abounding confusion concerning the representation invariance mechanisms of the two sophisticated systems. To investigate their relationship under common conditions, we proposed a representation invariance analysis approach based on data augmentation technology. Firstly, the original image library was expanded by data augmentation. The representation invariances of CNNs and the ventral visual stream were then studied by comparing the similarities of the corresponding layer features of CNNs and the prediction performance of visual encoding models based on functional magnetic resonance imaging (fMRI) before and after data augmentation. Our experimental results suggest that the architecture of CNNs, combinations of convolutional and fully-connected layers, developed representation invariance of CNNs. Remarkably, we found representation invariance belongs to all successive stages of the ventral visual stream. Hence, the internal correlation between CNNs and the human visual system in representation invariance was revealed. Our study promotes the advancement of invariant representation of computer vision and deeper comprehension of the representation invariance mechanism of human visual information processing.

Latest Advances in Human Brain Dynamics

“…Thus, representation invariance plays a typical role in CNN and human visual processing information under complicated image-based tasks. To investigate this relationship between CNNs and the human visual system, Cui et al [ 4 ] explored the representation invariances of CNNs and the ventral visual stream by comparing features from the layers of CNNs and the prediction performance of visual encoding models. This novel study untangled the importance of invariant representation of computer vision and the deeper conception of the representation invariant mechanism of the human visual information processing.…”