Binary Neural Networks for Memory-Efficient and Effective Visual Place Recognition in Changing Environments

Ferrarini, Bruno; Milford, Michael; McDonald-Maier, Klaus; Ehsan, Shoaib

doi:10.1109/tro.2022.3148908

Cited by 12 publications

(10 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Quantization is a viable strategy for efficient VPR, focusing on minimizing the precision of weights and activations. In its most extreme form, binary neural networks represent weights and activations with a single bit, yielding impressive memory and latency efficiencies [27], [28], thereby rendering them suitable for micro-controllers and low-power CPUs.…”

Section: Related Workmentioning

confidence: 99%

Highly-Efficient Binary Neural Networks for Visual Place Recognition

Ferrarini

Milford

McDonald-Maier

et al. 2022

2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

View full text Add to dashboard Cite

Visual Place Recognition (VPR) is a critical task for performing global re-localization in visual perception systems. It requires the ability to accurately recognize a previously visited location under variations such as illumination, occlusion, appearance and viewpoint. In the case of robotic systems and augmented reality, the target devices for deployment are battery powered edge devices. Therefore whilst the accuracy of VPR methods is important so too is memory consumption and latency. Recently new works have focused on the recall@1 metric as a performance measure with limited focus on resource utilization. This has resulted in methods that use deep learning models too large to deploy on low powered edge devices. We hypothesize that these large models are highly over-parameterized and can be optimized to satisfy the constraints of a low powered embedded system whilst maintaining high recall performance. Our work studies the impact of compact convolutional network architecture design in combination with full-precision and mixed-precision posttraining quantization on VPR performance. Importantly we not only measure performance via the recall@1 score but also measure memory consumption and latency. We characterize the design implications on memory, latency and recall scores and provide a number of design recommendations for VPR systems under these resource limitations.

show abstract

Section: Related Workmentioning

confidence: 99%

Highly-Efficient Binary Neural Networks for Visual Place Recognition

Ferrarini

Milford

McDonald-Maier

et al. 2022

2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Region-VLAD [54] is based on the same approach as Cross-Region-BoW but employs VLAD for feature pooling. [55] and [56] present computationally efficient and compact binary neural networks (BNN) for VPR achieving comparable performance in changing environments with full-precision systems such as HybridNet. However, BNNs require dedicated hardware or an inference engine that enables an efficient computation of bitwise operations.…”

Section: Hmm Sequencementioning

confidence: 99%

Sequence-Based Filtering for Visual Route-Based Navigation: Analyzing the Benefits, Trade-Offs and Design Choices

et al. 2022

Self Cite

View full text Add to dashboard Cite

Visual Place Recognition (VPR) is the ability to correctly recall a previously visited place using visual information under environmental, viewpoint and appearance changes. An emerging trend in VPR is the use of sequence-based filtering methods on top of single-frame-based place matching techniques for route-based navigation. The combination leads to varying levels of potential place matching performance boosts at increased computational costs. This raises a number of interesting research questions: How does performance boost (due to sequential filtering) vary along the entire spectrum of single-frame-based matching methods? How does sequence matching length affect the performance curve? Which specific combinations provide a good trade-off between performance and computation? However, there is lack of previous work looking at these important questions and most of the sequence-based filtering work to date has been used without a systematic approach. To bridge this research gap, this paper conducts an in-depth investigation of the relationship between the performance of single-frame-based place matching techniques and the use of sequence-based filtering on top of those methods. It analyzes individual trade-offs, properties and limitations for different combinations of single-frame-based and sequential techniques. The experiments conducted in this study demonstrate the benefits of sequence-based filtering over the single-frame-based approach using various VPR techniques. We found that applying sequence-based filtering to a lightweight descriptor can enable higher VPR accuracy than state-of-the-art methods such as NetVLAD, while running in shorter time. For example, matching a sequence of 16 images, CALC descriptor outperforms NetVLAD on Campus Loop dataset while taking about 22% less time to perform VPR.

show abstract

“…In [41], the authors have used the AlexNet ConvNet [42] pre-trained on the ImageNet ILSVRC dataset [43] for object recognition. The authors of [44] and [45] use binary neural networks (BNNs) for VPR. These systems are less computationally demanding than other CNN-based VPR techniques, while achieving similar place matching performance as full-precision systems.…”

Section: Literature Reviewmentioning

confidence: 99%

Data Efficient Visual Place Recognition Using Extremely JPEG-Compressed Images

Tomita¹,

Ferrarini²,

Milford³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Visual Place Recognition (VPR) is the ability of a robotic platform to correctly interpret visual stimuli from its on-board cameras in order to determine whether it is currently located in a previously visited place, despite different viewpoint, illumination and appearance changes. JPEG is a widely used image compression standard that is capable of significantly reducing the size of an image at the cost of image clarity. For applications where several robotic platforms are simultaneously deployed, the visual data gathered must be transmitted remotely between each robot. Hence, JPEG compression can be employed to drastically reduce the amount of data transmitted over a communication channel, as working with limited bandwidth for VPR can be proven to be a challenging task. However, the effects of JPEG compression on the performance of current VPR techniques have not been previously studied. For this reason, this paper presents an indepth study of JPEG compression in VPR related scenarios. We use a selection of well-established VPR techniques on 8 datasets with various amounts of compression applied. We show that by introducing compression, the VPR performance is drastically reduced, especially in the higher spectrum of compression. To overcome the negative effects of JPEG compression on the VPR performance, we present a fine-tuned CNN which is optimized for JPEG compressed data and show that it performs more consistently with the image transformations detected in extremely compressed JPEG images.

show abstract

Binary Neural Networks for Memory-Efficient and Effective Visual Place Recognition in Changing Environments

Cited by 12 publications

References 59 publications

Highly-Efficient Binary Neural Networks for Visual Place Recognition

Highly-Efficient Binary Neural Networks for Visual Place Recognition

Sequence-Based Filtering for Visual Route-Based Navigation: Analyzing the Benefits, Trade-Offs and Design Choices

Data Efficient Visual Place Recognition Using Extremely JPEG-Compressed Images

Contact Info

Product

Resources

About