Assessing Box Merging Strategies and Uncertainty Estimation Methods in Multimodel Object Detection

Abstract

This paper examines the impact of different box merging strategies for sampling-based uncertainty estimation methods in object detection. Also, a comparison between the almost exclusively used softmax confidence scores and the predicted variances on the quality of the final predictions estimates is presented. The results suggest that estimated variances are a stronger predictor for the detection quality. However, variance-based merging strategies do not improve significantly over the confidence-based alternative for the given setup. In contrast, we show that different methods to estimate the uncertainty of the predictions have a significant influence on the quality of the ensembling outcome. Since mAP does not reward uncertainty estimates, such improvements were only noticeable on the resulting PDQ scores.

LiDAR-MIMO: Efficient Uncertainty Estimation for LiDAR-based 3D Object Detection

“…To merge predictions, we used the mean method. Roza et al [30] implemented several different box merging strategies, and showed significant variations on sample-based uncertainty estimates resulting from different strategies. Based on their results, we used the mean method, which had the best performance overall.…”