Transfer learning may explain pigeons’ ability to detect cancer in histopathology

Abstract

Pigeons' unexpected competence in learning to categorize unseen histopathological images has remained an unexplained discovery for almost a decade. Could it be that knowledge transferred from their bird's-eye views of the earth's surface gleaned during flight contributes to this ability? Employing a simulation-based verification strategy, we re-capitulate this biological phenomenon with a machine-learning analog. We model pigeons' visual experience during flight with the self-supervised pre-training of a deep neural network on BirdsEyeViewNet (BEVNet); our large-scale aerial imagery dataset. As an analog of the differential food reinforcement performed in the Levenson et al. study, we apply transfer learning from this pre-trained model to the same Hematoxylin and Eosin H&E histopathology and radiology images and tasks the pigeons were trained and tested on. The study demonstrates that pre-training neural networks with bird's-eye view data results in close agreement with pigeon performance. These results support transfer learning as a reasonable computational model of pigeon representation learning. This is further validated with six large-scale downstream classification tasks using H&E stained whole slide image (WSI) data sets representing diverse cancer types.