Unsupervised deep learning based ego motion estimation with a downward facing camera

Gilles, Maximilian; Ibrahimpasic, Sascha

doi:10.1007/s00371-021-02345-6

Cited by 6 publications

(4 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Camera motion estimation is one of the most important technologies in many applications, such as automatic driving and assistive technologies. In particular, motion estimation by pointing a camera at a road surface has an advantage in that objects that can affect it, such as automobiles and pedestrians, are usually not easily visible [1][2][3][4][5][6]. Teshima et al [1] proposed a method to estimate the position and orientation of a vehicle from sequential images of the ground using the known homography obtained from a normal camera.…”

Section: Introduction 1backgroundsmentioning

confidence: 99%

“…Saurer et al [2] proposed a method to find different minimal solutions for egomotion estimation of a camera based on homography knowing the gravity vector between images. Gilles et al [3] proposed a method based on unsupervised deep learning for motion estimation with a downward-looking camera. However, in spite of active research, camera motion estimation by using the road surface has the following problems: it is difficult to extract and match feature points robustly from noisy ground textures; the high-speed movement of cameras causes motion blur; it often involves challenging illumination conditions.…”

Section: Introduction 1backgroundsmentioning

confidence: 99%

See 1 more Smart Citation

Accuracy and Speed Improvement of Event Camera Motion Estimation Using a Bird’s-Eye View Transformation

Ozawa

Sekikawa

Saitō

2022

Sensors

View full text Add to dashboard Cite

Event cameras are bio-inspired sensors that have a high dynamic range and temporal resolution. This property enables motion estimation from textures with repeating patterns, which is difficult to achieve with RGB cameras. Therefore, motion estimation of an event camera is expected to be applied to vehicle position estimation. An existing method, called contrast maximization, is one of the methods that can be used for event camera motion estimation by capturing road surfaces. However, contrast maximization tends to fall into a local solution when estimating three-dimensional motion, which makes correct estimation difficult. To solve this problem, we propose a method for motion estimation by optimizing contrast in the bird’s-eye view space. Instead of performing three-dimensional motion estimation, we reduced the dimensionality to two-dimensional motion estimation by transforming the event data to a bird’s-eye view using homography calculated from the event camera position. This transformation mitigates the problem of the loss function becoming non-convex, which occurs in conventional methods. As a quantitative experiment, we created event data by using a car simulator and evaluated our motion estimation method, showing an improvement in accuracy and speed. In addition, we conducted estimation from real event data and evaluated the results qualitatively, showing an improvement in accuracy.

show abstract

Section: Introduction 1backgroundsmentioning

confidence: 99%

Section: Introduction 1backgroundsmentioning

confidence: 99%

Accuracy and Speed Improvement of Event Camera Motion Estimation Using a Bird’s-Eye View Transformation

Ozawa

Sekikawa

Saitō

2022

Sensors

View full text Add to dashboard Cite

show abstract

“…The network returns the 6-DOF camera pose, and implicitly learns the scale factor without having knowledge about the intrinsic camera parameters. A promising unsupervised approach 21 for VO with a downward-facing camera was proposed. However, the used loss function was not suitable for avoiding small errors for the rotational accuracy.…”

Section: Introductionmentioning

confidence: 99%

Optical encoder neural network: a CNN-based optical encoder for robot localization

et al. 2022

View full text Add to dashboard Cite

.We tackle the problem of robot localization by means of a deep learning (DL) approach. Convolutional neural networks, mainly devised for image analysis, are at the core of the proposed solution. An optical encoder neural network (OE-net) is devised to give back the relative pose of a robot by processing consecutive images. A monocular camera, fastened on the robot and oriented toward the floor, collects the vision data. The OE-net takes a pair of the acquired consecutive images as input and provides the relative pose information. The neural network is trained using a supervised learning approach. This preliminary study, made on synthetic images, suggests that a convolutional network and hence a DL approach can be a viable complement to the traditional visual odometry for robot ego-motion estimation. The obtained outcomes look very promising.

show abstract

“…Appearance-based approaches like that of Zaman [2007] and Gilles and Ibrahimpasic [2021] can perform relative localization by estimating transformations between ground images directly based on the observed appearance changes. For example, in the case of Gilles and Ibrahimpasic [2021], using a deep neural network that is trained in an unsupervised manner for image registration.…”

Section: Relative Localizationmentioning

confidence: 99%

Ground texture based localization

Schmid

View full text Add to dashboard Cite

This dissertation is concerned with the task of map-based self-localization, using images of the ground recorded with a downward-facing camera. In this context, map-based (self-)localization is the task of determining the position and orientation of a query image that is to be localized. The map used for this purpose consists of a set of reference images with known positions and orientations in a common coordinate system. For localization, the considered methods determine correspondences between features of the query image and those of the reference images. In comparison with localization approaches that use images of the surrounding environment, we expect that using images of the ground has the advantage that, unlike the surrounding, the visual appearance of the ground is often long-term stable. Also, by using active lighting of the ground, localization becomes independent of external lighting conditions. This dissertation includes content of several published contributions, which present research on the development and testing of methods for feature-based localization of ground images. Our first contribution examines methods for the extraction of image features that have not been designed to be used on ground images. This survey shows that, with appropriate parametrization, several of these methods are well suited for the task. Based on this insight, we develop and examine methods for various subtasks of map-based localization in the following contributions. We examine global localization, where all reference images have to be considered, as well as local localization, where an approximation of the query image position is already known, which allows for disregarding reference images with a large distance to this position. In our second contribution, we present the first systematic comparison of state-of-the-art methods for ground texture based localization. Furthermore, we present a method, which is characterized by its usage of our novel feature matching technique. This technique is called identity matching, as it matches only those features with identical descriptors, in contrast to the state-of-the-art that also matches features with similar descriptors. We show that our method is well suited for global and local localization, as it has favorable scaling with the number of reference images considered during the localization process. In another contribution, we develop a variant of our localization method that is significantly faster to compute, as it applies a sampling approach to determine the image positions at which local features are extracted, instead of using classical feature detectors. Two further contributions are concerned with global localization. The first one introduces a prediction model for the global localization performance, based on an evaluation of the local localization performance. This allows us to quickly evaluate any considered parameter settings of global localization methods. The second contribution introduces a learning-based method that computes compact descriptors of ground images. This descriptor can be used to retrieve the overlapping reference images of a query image from a large set of reference images with little computational effort. The most recent contribution included in this dissertation presents a new ground image database, which was recorded with a dedicated platform using a downward-facing camera. In addition to the data, we also explain our guidelines for the construction of the platform. In comparison with existing databases, our database contains more images and presents a larger variety of ground textures. Furthermore, this database enables us to perform the first systematic evaluation of how localization performance is affected by the time interval between the point in time at which the reference images are recorded and the point in time at which the query image is recorded. We find out that for outdoor areas all ground texture based localization methods have reliability issues, if the time interval between the recording of the query and reference images is large, and also if there are different weather conditions. These findings point to remaining challenges in ground texture base localization that should be addressed in future work.

show abstract

Unsupervised deep learning based ego motion estimation with a downward facing camera

Cited by 6 publications

References 22 publications

Accuracy and Speed Improvement of Event Camera Motion Estimation Using a Bird’s-Eye View Transformation

Accuracy and Speed Improvement of Event Camera Motion Estimation Using a Bird’s-Eye View Transformation

Optical encoder neural network: a CNN-based optical encoder for robot localization

Ground texture based localization

Contact Info

Product

Resources

About