SplitNets: Designing Neural Architectures for Efficient Distributed Computing on Head-Mounted Systems

Abstract

We design deep neural networks (DNNs) and corresponding networks' splittings to distribute DNNs' workload to camera sensors and a centralized aggregator on head mounted devices to meet system performance targets in inference accuracy and latency under the given hardware resource constraints. To achieve an optimal balance among computation, communication, and performance, a splitaware neural architecture search framework, SplitNets, is introduced to conduct model designing, splitting, and communication reduction simultaneously. We further extend the framework to multi-view systems for learning to fuse inputs from multiple camera sensors with optimal performance and systemic efficiency. We validate SplitNets for singleview system on ImageNet as well as multi-view system on 3D classification, and show that the SplitNets framework achieves state-of-the-art (SOTA) performance and system latency compared with existing approaches.

Flexible collaborative inference for computer vision

“…Finally, in the case of splitting NNs, trends from offloading can carry over to single-device multi-processor architectures. An emerging approach is the use of a nearsensor unit with pre-processing and compression layers (the encoder) and a central unit processing the bulk of computation (the decoder) (GOMEZ et al, 2022;DONG et al, 2022). According to Abrash (2021), this is pointed as key HW/SW evolution for enabling AR devices because the communication between sensor and compute elements can be the most significant factor in energy consumption.…”

“…Compression is usually achieved via the addition of an extraneous reduction component, which invalidates the assumption of a fixed architecture. One proposed solution is to use the ratio between encoder and decoder as a metric (DONG et al, 2022), as the tradeoff between computation and precision in the whole model should be monotonic.…”