Learned Camera Gain and Exposure Control for Improved Visual Feature Detection and Matching

Tomasi, Justin; Wagstaff, Brandon; Waslander, Steven L.; Kelly, Jonathan

doi:10.1109/lra.2021.3058909

Cited by 22 publications

(8 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Indeed, this is similar to the performance obtained by Shim [ 16 ], who reported achieving 70 Hz using an Intel Core i5-6260U (1.80 GHz) for a downsized image resolution of 320 × 240 pixels. While Tomasi [ 20 ] was able to reach a processing rate of 640 Hz using a trained CNN, the algorithm was implemented on an NVIDIA GeForce GTX 1050 Ti GPU. Reimplementing the proposed algorithm on a GPU (which are known to be 1–2 orders of magnitude faster than CPUs for image processing) would likely yield a similar processing rate.…”

Section: Discussionmentioning

confidence: 99%

“…More recently, Tomasi [ 20 ] also proposed to directly maximize the number of detected and successfully matched features across frames as a proxy for VO performance through a self-supervised learning method. However, from the current state of the literature, it remains unclear how these metrics compare to one another as substitutes of VO performance, since direct cross-correlation analyses are rarely offered.…”

Section: Introductionmentioning

confidence: 99%

“…Instead, predictive AE algorithms use a model to predict the effect of future exposure parameters on the optimization metric. For learned control policies (e.g., [ 14 , 17 , 20 ]), a predictive model is implicitly embedded in the policy. In contrast, explicit predictive methods offer more interpretability, and leverage known information about the camera’s image acquisition process.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Auto-Exposure Algorithm for Enhanced Mobile Robot Localization in Challenging Light Conditions

Bégin

Hunter

2022

Sensors

View full text Add to dashboard Cite

The success of robot localization based on visual odometry (VO) largely depends on the quality of the acquired images. In challenging light conditions, specialized auto-exposure (AE) algorithms that purposely select camera exposure time and gain to maximize the image information can therefore greatly improve localization performance. In this work, an AE algorithm is introduced which, unlike existing algorithms, fully leverages the camera’s photometric response function to accurately predict the optimal exposure of future frames. It also features feedback that compensates for prediction inaccuracies due to image saturation and explicitly balances motion blur and image noise effects. For validation, stereo cameras mounted on a custom-built motion table allow different AE algorithms to be benchmarked on the same repeated reference trajectory using the stereo implementation of ORB-SLAM3. Experimental evidence shows that (1) the gradient information metric appropriately serves as a proxy of indirect/feature-based VO performance; (2) the proposed prediction model based on simulated exposure changes is more accurate than using γ transformations; and (3) the overall accuracy of the estimated trajectory achieved using the proposed algorithm equals or surpasses classic exposure control approaches. The source code of the algorithm and all datasets used in this work are shared openly with the robotics community.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Auto-Exposure Algorithm for Enhanced Mobile Robot Localization in Challenging Light Conditions

Bégin

Hunter

2022

Sensors

View full text Add to dashboard Cite

show abstract

“…Although deep learning has made significant achievements in image enhancement and camera control [12][13][14][15][16][17], Fig. 2.…”

Section: Introductionmentioning

confidence: 99%

Learning to Generate High-Quality Images for Homography Estimation

Lin

Zhao

2022

2022 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Nowadays, robots have gradually replaced humans to perform tasks in challenging environments, but visually satisfactory images may not always contain high-quality features required by downstream algorithms. In this paper, we reveal that in the homography estimation, which is the essential task of robot vision, algorithms show different sensitivities to aerial images and sometimes perform better in over/under-exposed scenes. To this point, we utilize the gamma correction theory to design an interpretable exposure adjustment function, and train a personalized exposure selection network ESH-Net by contrastive learning. After adjusting the exposure of original input by our method, algorithms can obtain better homography results. Comprehensive experiments are conducted on samples generated from the aerial DOTA dataset. Results show that our method can produce high-quality images for both learning-based and traditional homography estimators, and outperforms other exposure enhancement networks.

show abstract

“…Experimental evidence for improved pose estimation accuracy with a larger number of features is provided in[19].…”

mentioning

confidence: 99%

Learned Camera Gain and Exposure Control for Improved Visual Feature Detection and Matching

Tomasi,

Wagstaff,

Waslander

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Successful visual navigation depends upon capturing images that contain sufficient useful information. In this paper, we explore a data-driven approach to account for environmental lighting changes, improving the quality of images for use in visual odometry (VO) or visual simultaneous localization and mapping (SLAM). We train a deep convolutional neural network model to predictively adjust camera gain and exposure time parameters such that consecutive images contain a maximal number of matchable features. The training process is fully selfsupervised: our training signal is derived from an underlying VO or SLAM pipeline and, as a result, the model is optimized to perform well with that specific pipeline. We demonstrate through extensive real-world experiments that our network can anticipate and compensate for dramatic lighting changes (e.g., transitions into and out of road tunnels), maintaining a substantially higher number of inlier feature matches than competing camera parameter control algorithms.

show abstract

Learned Camera Gain and Exposure Control for Improved Visual Feature Detection and Matching

Cited by 22 publications

References 18 publications

Auto-Exposure Algorithm for Enhanced Mobile Robot Localization in Challenging Light Conditions

Auto-Exposure Algorithm for Enhanced Mobile Robot Localization in Challenging Light Conditions

Learning to Generate High-Quality Images for Homography Estimation

Learned Camera Gain and Exposure Control for Improved Visual Feature Detection and Matching

Contact Info

Product

Resources

About