Multisensory embedded pose estimation

Abstract

We present a multisensory method for estimating the transformation of a mobile phone between two images taken from its camera. Pose estimation is a necessary step for applications such as 3D reconstruction and panorama construction, but detecting and matching robust features can be computationally expensive. In this paper we propose a method for combining the inertial sensors (accelerometers and gyroscopes) of a mobile phone with its camera to provide a fast and accurate pose estimation.We use the inertial based pose to warp two images into the same perspective frame. We then employ an adaptive FAST feature detector and image patches, normalized with respect to illumination, as feature descriptors. After the warping the images are approximately aligned with each other so the search for matching key-points also becomes faster and in certain cases more reliable. Our results show that by incorporating the inertial sensors we can considerably speed up the process of detecting and matching key-points between two images, which is the most time consuming step of the pose estimation.

Depth-assisted rectification for real-time object detection and pose estimation

“…The methods described in [13,28,57,58] first projectively rectify the whole image and then detect invariant features on the normalized result, while the DARP method does the opposite. In addition, [57] is designed for offline 3D reconstruction, [13,28,58] target only planar scenes and [13,28] require an inertial sensor.…”

“…In addition, [57] is designed for offline 3D reconstruction, [13,28,58] target only planar scenes and [13,28] require an inertial sensor.…”

“…Methods that use a 3D reconstruction of the scene often rely on texture based local descriptors and are not adequate for texture-less objects [27,38,57,58]. There are also some approaches that require the presence of inertial sensors [13,28]. The DARC method does not need any additional sensor besides an RGB-D camera and is based on normalization of contour features, allowing pose estimation of texture-less planar targets.…”

Linking Computer Vision with Off-the-Shelf Accelerometry through Kinetic Energy for Precise Localization

“…working independently at separate time slots, or prioritizing computer vision and relying on INS when the cameras fail to deliver information) are described in [19] and references therein. In this sense, recent examples include navigation systems integrating cameras, gyroscopes and accelerometers, combining the data with an extended Kalman filter [20], an unscented Kalman filter [21], or the employment of Bayesian segmentation to detect a moving person (with "Mixtures of Gaussians" for background modeling), and a particle filter to track the person in the scene [22].…”

Depth-Assisted Rectification of Patches - Using RGB-D Consumer Devices to Improve Real-time Keypoint Matching

“…The methods described in (Eyjolfsdottir and Turk, 2011), (Kurz and Benhimane, 2011), (Wu et al, 2008) and (Yang et al, 2010) first projectively rectify the whole image and then detect invariant features on the normalized result, while the DARP method does the opposite. In addition, (Wu et al, 2008) is designed for offline 3D reconstruction, (Eyjolfsdottir and Turk, 2011), (Kurz and Benhimane, 2011) and (Yang et al, 2010) target only planar scenes and (Eyjolfsdottir and Turk, 2011) and (Kurz and Benhimane, 2011) require an inertial sensor. Concurrent with this research (Marcon et al, 2012) used an RGB-D sensor to perform patch rectification using PCA, followed by 2D Fourier-Mellin Transform for description.…”