Bridging the appearance gap: Multi-experience localization for long-term visual teach and repeat

Paton, Michael; MacTavish, Kirk; Warren, Michael; Barfoot, Timothy D.

doi:10.1109/iros.2016.7759303

Cited by 68 publications

(41 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…b) UTIAS In The Dark: The UTIAS In The Dark (InTheDark) dataset [3] provides stereo imagery of a 250 m outdoor loop traversed repeatedly over a 30-hour period using on-board headlights to illuminate the scene at night. We use the multi-experience localization system in [3] to obtain corresponding image pairs with overlapping fields of view but non-identical poses. We train our models using leftcamera images from 66 traversals and test on 7 held-out traversals spanning a full day-night cycle (listed in Table I).…”

Section: A Datasetsmentioning

confidence: 99%

“…2 Long-term maps based on multiple visual 'experiences' of an environment have proven to be effective tools for metric localization through daily and seasonal appearance change [1]- [4]. In [3], consecutive visual experiences are recorded in a spatio-temporal pose graph, and localization against a privileged mapping experience proceeds by recalling a relevant experience and tracing through a chain of relative transformations in the graph. This process is often aided by a prior on the vehicle's topological location in the graph, whether from dead reckoning, place recognition, or GNSS, which serves to limit the number of candidate vertices for metric localization.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning Matchable Image Transformations for Long-Term Metric Visual Localization

Clement

Gridseth²,

Tomasi

et al. 2020

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Long-term metric self-localization is an essential capability of autonomous mobile robots, but remains challenging for vision-based systems due to appearance changes caused by lighting, weather, or seasonal variations. While experience-based mapping has proven to be an effective technique for bridging the 'appearance gap,' the number of experiences required for reliable metric localization over days or months can be very large, and methods for reducing the necessary number of experiences are needed for this approach to scale. Taking inspiration from color constancy theory, we learn a nonlinear RGB-to-grayscale mapping that explicitly maximizes the number of inlier feature matches for images captured under different lighting and weather conditions, and use it as a pre-processing step in a conventional single-experience localization pipeline to improve its robustness to appearance change. We train this mapping by approximating the target non-differentiable localization pipeline with a deep neural network, and find that incorporating a learned low-dimensional context feature can further improve cross-appearance feature matching. Using synthetic and realworld datasets, we demonstrate substantial improvements in localization performance across day-night cycles, enabling continuous metric localization over a 30-hour period using a single mapping experience, and allowing experience-based localization to scale to long deployments with dramatically reduced data requirements.

show abstract

Section: A Datasetsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Learning Matchable Image Transformations for Long-Term Metric Visual Localization

Clement

Gridseth²,

Tomasi

et al. 2020

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

show abstract

“…VT&R and similar route-based navigation algorithms have a rich history on ground platforms [1]- [4], with the most recent extension adapted to include multiple experiences, increasing the autonomous performance time from a few days to several months [5]. On UAVs, there are now several demonstrations of teach-and-repeat style algorithms from the authors of this paper and others [6]- [9].…”

Section: Previous Workmentioning

confidence: 99%

“…In this paper, we adapt the traditional VT&R methodology to suit these target use cases and apply our VT&R 2.0 system [5] on-board a multirotor UAV ( Fig. 1) to demonstrate closed-loop operation.…”

Section: Introductionmentioning

confidence: 99%

There's No Place Like Home: Visual Teach and Repeat for Emergency Return of Multirotor UAVs During GPS Failure

Warren

Greeff

Patel

et al. 2019

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

Redundant navigation systems are critical for safe operation of UAVs in high-risk environments. Since most commercial UAVs almost wholly rely on GPS, jamming, interference and multi-pathing are real concerns that usually limit their operations to low-risk environments and VLOS. This paper presents a vision-based route-following system for the autonomous, safe return of UAVs under primary navigation failure such as GPS jamming. Using a Visual Teach and Repeat framework to build a visual map of the environment during an outbound flight, we show the autonomous return of the UAV by visually localising the live view to this map when a simulated GPS failure occurs, controlling the vehicle to follow the safe outbound path back to the launch point. Using gimbal-stabilised stereo vision alone, without reliance on external infrastructure or inertial sensing, Visual Odometry and localisation are achieved at altitudes of 5-25 m and flight speeds up to 55 km/h. We examine the performance of the visual localisation algorithm under a variety of conditions and also demonstrate closed-loop autonomy along a complicated 450 m path.

show abstract

“…Our results show that by selecting 20-30% of landmarks using our proposed approach, a similar localization performance as the baseline strategy using all landmarks is achieved. promising approach to address this problem has been proposed in the form of multisession maps (Churchill & Newman, 2013;Mühlfellner et al, 2016;Paton, Mactavish, Warren, & Barfoot, 2016) that incorporate visual cues from more than one appearance condition. The resulting maps, however, quickly grow in size and become impractical to handle on the mobile robotic platform.…”

mentioning

confidence: 99%

Appearance‐based landmark selection for visual localization

Bürki

Cadena

Gilitschenski³

et al. 2019

Journal of Field Robotics

View full text Add to dashboard Cite

show abstract

Bridging the appearance gap: Multi-experience localization for long-term visual teach and repeat

Cited by 68 publications

References 18 publications

Learning Matchable Image Transformations for Long-Term Metric Visual Localization

Learning Matchable Image Transformations for Long-Term Metric Visual Localization

There's No Place Like Home: Visual Teach and Repeat for Emergency Return of Multirotor UAVs During GPS Failure

Appearance‐based landmark selection for visual localization

Contact Info

Product

Resources

About