Shape-Biased Learning by Thinking Inside the Box

Abstract

Deep Neural Networks (DNNs) may surpass human-level performance on vision tasks such as object recognition and detection, but their model behavior still differs from human behavior in important ways. One prominent example of this difference, and the main focus of our paper, is that DNNs trained on ImageNet exhibit an object texture bias, while humans are consistently biased towards shape. DNN shape-bias can be increased by data augmentation, but next to being computationally more expensive, data augmentation is a biologically implausible method of creating texture-invariance. We present an empirical study on texture-shape-bias in DNNs showcasing that high texture-bias correlates with high background-object ratio. In addition, DNNs trained on tight object bounding boxes of ImageNet images are sub-stantially more biased towards shape than models trained on the full images. Using a custom dataset of high-resolution, object annotated scene images, we show that (I) shape-bias systematically varies with training on bounding boxes, (II) removal of global object shape as a result of commonly applied cropping during training increases texture bias, (III) shape-bias is negatively correlated with test accuracy on ImageNet while being positively correlated on cue-conflict images created using bounding boxes, following the trend of humans. Overall, we show that an improved supervision signal that better reflects the visual features that truly belong to the to-be-classified object increases the shape-bias of deep neural networks. Our results also imply that simultaneous human alignment on both classification accuracy and strategy can not be achieved on default ImageNet images, suggesting the need for new assessments of both shape-bias and behavioural alignment between DNNs and humans.