A CNN-SIFT Hybrid Pedestrian Navigation Method Based on First-Person Vision

Abstract

Abstract:The emergence of new wearable technologies, such as action cameras and smart glasses, has driven the use of the first-person perspective in computer applications. This field is now attracting the attention and investment of researchers aiming to develop methods to process first-person vision (FPV) video. The current approaches present particular combinations of different image features and quantitative methods to accomplish specific objectives, such as object detection, activity recognition, user-machine interaction, etc. FPV-based navigation is necessary in some special areas, where Global Position System (GPS) or other radio-wave strength methods are blocked, and is especially helpful for visually impaired people. In this paper, we propose a hybrid structure with a convolutional neural network (CNN) and local image features to achieve FPV pedestrian navigation. A novel end-to-end trainable global pooling operator, called AlphaMEX, has been designed to improve the scene classification accuracy of CNNs. A scale-invariant feature transform (SIFT)-based tracking algorithm is employed for movement estimation and trajectory tracking of the person through each frame of FPV images. Experimental results demonstrate the effectiveness of the proposed method. The top-1 error rate of the proposed AlphaMEX-ResNet outperforms the original ResNet (k = 12) by 1.7% on the ImageNet dataset. The CNN-SIFT hybrid pedestrian navigation system reaches 0.57 m average absolute error, which is an adequate accuracy for pedestrian navigation. Both positions and movements can be well estimated by the proposed pedestrian navigation algorithm with a single wearable camera.

A Coarse-to-Fine Framework for Multiple Pedestrian Crossing Detection

“…With the advent of smart phones and location-based services, there has been an increased interest in providing route guidance to pedestrians under different environments [1][2][3]. As an important part of transportation infrastructures, pedestrian crossing serves to secure pedestrians' lives and possessions and keep traffic flow in order [4].…”

A Novel Pedestrian Dead Reckoning Algorithm for Multi-Mode Recognition Based on Smartphones

“…Currently, pedestrian indoor navigation and positioning can be achieved using two types of methods. The first type of method is based on wireless technologies, e.g., WiFi [3][4][5], ultra-wideband (UWB) [6,7], visual sensors [8], radio frequency identification (RFID) [9], ibeacon [10], Bluetooth, and/or ZigBee, with a multi-source information fusion technique [11] to obtain pedestrian location information. For the first type of method, its location errors do not accumulate over time.…”

Integrated Velocity Measurement Algorithm Based on Optical Flow and Scale-Invariant Feature Transform

“…Reference [17] used a SIFT algorithm to recognize landmarks and an extended Kalman filter to update the position; use of the A-star algorithm for path planning and the virtual force method for autonomous navigation of a mobile robot was then proposed, and finally, the effectiveness and applicability of the proposed method were demonstrated experimentally using the mobile robot. Reference [18] proposed a hybrid structure containing a convolutional neural network (CNN) and local image features to achieve first-person vision (FPV) pedestrian navigation, where a SIFT-based tracking algorithm was used to estimate motion and track each frame of the FPV image. To obtain higher homing precision for local robot navigation, a novel robot visual homing algorithm was proposed that combined SIFT features with the warping method [19], and the effectiveness of the proposed algorithm was verified experimentally.…”