Ventral-Dorsal Neural Networks: Object Detection Via Selective Attention

Abstract

Deep Convolutional Neural Networks (CNNs) have been repeatedly proven to perform well on image classification tasks. Object detection methods, however, are still in need of significant improvements. In this paper, we propose a new framework called Ventral-Dorsal Networks (VDNets) which is inspired by the structure of the human visual system. Roughly, the visual input signal is analyzed along two separate neural streams, one in the temporal lobe and the other in the parietal lobe. The coarse functional distinction between these streams is between object recognition -the "what" of the signal -and extracting location related information -the "where" of the signal. The ventral pathway from primary visual cortex, entering the temporal lobe, is dominated by "what" information, while the dorsal pathway, into the parietal lobe, is dominated by "where" information. Inspired by this structure, we propose the integration of a "Ventral Network" and a "Dorsal Network", which are complementary. Information about object identity can guide localization, and location information can guide attention to relevant image regions, improving object recognition. This new dual network framework sharpens the focus of object detection. Our experimental results reveal that the proposed method outperforms state-of-the-art object detection approaches on PASCAL VOC 2007 by 8% (mAP) and PASCAL VOC 2012 by 3% (mAP). Moreover, a comparison of techniques on Yearbook images displays substantial qualitative and quantitative benefits of VDNet. Figure 1. Primary visual cortex and two processing streams. The "ventral stream" projects into the temporal lobe, and the "dorsal stream" extends into the parietal lobe. These interacting pathways inspire the use of a "Ventral Net" and a "Dorsal Net".

Automotive Radar Processing With Spiking Neural Networks: Concepts and Challenges

“…Object detection and localization is also performed in the visual cortex in the ventral and dorsal stream (Desimone and Duncan, 1995). Artificial neural networks like CNNs have taken inspiration from that and are now highly-performant for this task (Ebrahimpour et al, 2019). In such approach, several radar processing steps (target detection, clustering and classification) can be realized by a single artificial neural network as demonstrated by Pérez et al (2019).…”

Emergence of Double-slit Interference by Representing Visual Space in Artificial Neural Networks

“…Inspired by separate pathways in the visual cortex, Ebrahimpour et al [12] proposed a ventral-dorsal network for detecting objects. The Ventral Net removes backgrounds using a selective activation cue.…”

Why Do Self-Supervised Models Transfer? Investigating the Impact of Invariance on Downstream Tasks

“…The ventral stream deals with the "what" of object recognition; and the dorsal stream deals with the "where" of spatial and motion information. This decomposition into specialised models has been exploited in applications such as object detection [17] and semantic grasping [28] in robotics. At the intersection of neuroscience and self-supervised learning, [2] showed that a two branch neural network trained with the CPC [41] loss on videos leads to dorsal and ventrallike pathways emerging.…”