Network-Aware 5G Edge Computing for Object Detection: Augmenting Wearables to “See” More, Farther and Faster

Abstract

Advanced wearable devices are increasingly incorporating high-resolution multi-camera systems. As state-of-the-art neural networks for processing the resulting image data are computationally demanding, there has been a growing interest in leveraging fifth generation (5G) wireless connectivity and mobile edge computing for offloading this processing closer to end-users. To assess this possibility, this paper presents a detailed simulation and evaluation of 5G wireless offloading for object detection in the case of a powerful, new smart wearable called VIS 4 ION, for the Blind-and-Visually Impaired (BVI). The current VIS 4 ION system is an instrumented book-bag with high-resolution cameras, vision processing, and haptic and audio feedback. The paper considers uploading the camera data to a mobile edge server to perform real-time object detection and transmitting the detection results back to the wearable. To determine the video requirements, the paper evaluates the impact of video bit rate and resolution on object detection accuracy and range. A new street scene dataset with labeled objects relevant to BVI navigation is leveraged for analysis. The vision evaluation is combined with a full-stack wireless network simulation to determine the distribution of throughputs and delays with real navigation paths and ray-tracing from new high-resolution 3D models in an urban environment. For comparison, the wireless simulation considers both a standard 4G-Long Term Evolution (LTE) sub-6-GHz carrier and high-rate 5G millimeter-wave (mmWave) carrier. The work thus provides a thorough and detailed assessment of edge computing for object detection with mmWave and sub-6-GHz connectivity in an application with both high bandwidth and low latency requirements.INDEX TERMS Mobile edge computing, millimeter-wave, 5G wireless, smart wearables, mobile machine vision, deep learning, object detection.(only up to 480 × 480). They also do not simulate real wireless networks.

Accuracy and Usability of Smartphone-Based Distance Estimation Approaches for Visual Assistive Technology Development

“…RGB images can be uploaded to the cloud for processing, with distances, objects, and potentially feedback reported back to the App. While cloud processing requires network connectivity and bandwidth, and adds network latency, it can still be beneficial for users in terms of battery or CPU usage, accuracy, and even overall latency (e.g., 84 ms) when local systems cannot perform the same computations within a time-efficient manner [11] . Finally, it should also be noted that our results reflect the performance of approaches available at the time of testing, and that further hardware or software revisions by the developers may alter their accuracy and usability metrics in the future.…”

Detect and Approach: Close-Range Navigation Support for People with Blindness and Low Vision

“…Such a system holds great promise for assisting pBLV in their daily living. Future research may develop a system where the video is uploaded to an edge server for conducting all computation tasks, to further reduce the wearable system weight [29].…”

Characterizing the response of piezoelectric-polymer composite haptic actuators and their interaction with skin

“…Despite the rapid increase of devices that include vibrotactile feedback, available vibration motors do not display adequate performance metrics for several critical haptics applications. In particular, electronic travel aids for blind and low vision people [5][6][7] and emergency response devices 8,9 require both complex representations of the environment (for example, through different vibration frequencies), along with fast activation and deactivation of the stimuli. Standard vibration motors for haptics, linear resonance actuators (LRAs) and eccentric rotating mass actuators (ERMs), cannot satisfy both of these requirements simultaneously.…”