An enhanced N-point interpolation method to eliminate average precision distortion

ZHANG, Haodi; Rogozan, Alexandrina; Bensrhair, Abdelaziz

doi:10.1016/j.patrec.2022.04.028

Cited by 4 publications

(7 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is avoided by preprocessing the precision recall curve to eliminate the zigzag behavior before estimating the AUC. Basically, there are two methods to do this: 11-point interpolation and all-point interpolation [48].…”

Section: P Recision = T P T P + F P (4)mentioning

confidence: 99%

“…The highest precision whose recall value is greater than a particular value is taken into consideration in this definition of AP rather than the precision seen at each recall level [48].…”

Section: P Recision = T P T P + F P (4)mentioning

confidence: 99%

“…Here, the AP is generated after interpolating the precision at each level, using the highest precision whose recall value is greater or equal to the particular value, as opposed to using the precision observed at only a few places [48].…”

Section: P Recision = T P T P + F P (4)mentioning

confidence: 99%

“…For a better understanding of the 11-point and all-point interpolations, let's take an example of an object detection case [48] whose precision and recall curve is shown in Fig. 5.…”

Section: P Recision = T P T P + F P (4)mentioning

confidence: 99%

See 3 more Smart Citations

Key Indicators to Assess the Performance of LiDAR-Based Perception Algorithms: A Literature Review

Karri,

da Silva,

Correia

2023

IEEE Access

View full text Add to dashboard Cite

Perception algorithms are essential for autonomous or semi-autonomous vehicles to perceive the semantics of their surroundings, including object detection, panoptic segmentation, and tracking. Decision-making in case of safety-critical situations, like autonomous emergency braking and collision avoidance, relies on the outputs of these algorithms. This makes it essential to correctly assess such perception systems before their deployment and to monitor their performance when in use. It is difficult to test and validate these systems, particularly at runtime, due to the high-level and complex representations of their outputs. This paper presents an overview of different existing metrics used for the evaluation of LiDAR-based perception systems, emphasizing particularly object detection and tracking algorithms due to their importance in the final perception outcome. Along with generally used metrics, we also discuss the impact of Planning KL-Divergence (PKL), Timed Quality Temporal Logic (TQTL), and Spatio-temporal Quality Logic (STQL) metrics on object detection algorithms. In the case of panoptic segmentation, Panoptic Quality (PQ) and Parsing Covering (PC) metrics are analysed resorting to some pretrained models. Finally, it addresses the application of diverse metrics to evaluate different pretrained models with the respective perception algorithms on publicly available datasets. Besides the identification of the various metrics being proposed, their performance and influence on models are also assessed after conducting new tests or reproducing the experimental results of the reference under consideration.

show abstract

Section: P Recision = T P T P + F P (4)mentioning

confidence: 99%

“…The highest precision whose recall value is greater than a particular value is taken into consideration in this definition of AP rather than the precision seen at each recall level [48].…”

Section: P Recision = T P T P + F P (4)mentioning

confidence: 99%

Section: P Recision = T P T P + F P (4)mentioning

confidence: 99%

“…For a better understanding of the 11-point and all-point interpolations, let's take an example of an object detection case [48] whose precision and recall curve is shown in Fig. 5.…”

Section: P Recision = T P T P + F P (4)mentioning

confidence: 99%

See 2 more Smart Citations

Key Indicators to Assess the Performance of LiDAR-Based Perception Algorithms: A Literature Review

Karri,

da Silva,

Correia

2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…A high AP indicates high values of Pre and Rec, meaning that the model is accurate. In practice, the PRC is a zig-zag line; thus, an interpolation method [39] is often utilized to facilitate the calculation.…”

Section: A Performance Metricsmentioning

confidence: 99%

An Enhanced YOLOv4 Model With Self-Dependent Attentive Fusion and Component Randomized Mosaic Augmentation for Metal Surface Defect Detection

2022

View full text Add to dashboard Cite

Metal surface quality control is significant in the production line of metal products. Detecting metal surface defects is challenging due to the various types and morphological patterns. Recent advances have witnessed deep learning-based automated optical inspection systems as a promising solution. This paper presents an enhanced YOLOv4 model for metal surface defect detection (MSDD). Specifically, we integrate three boosting components into the original YOLOv4, including 1) a self-dependent attentive fusion (SAF) block, placed within the model neck, to enhance inter-path and cross-layer feature fusion, 2) a component randomized Mosaic augmentation (CRMA) scheme to strategically discourage an overtransformed image to participate in training, and 3) a perturbation agnostic (PA) label smoothing method to keep the model from making over-confident predictions and thus act as a means of regularization. The proposed method has been validated on a self-developed MSDD dataset. It is shown that each boosting component can lead to an impressive mAP gain, and the final model outperforms the baselines,

show abstract

Local Evaluation of Large-scale Remote Sensing Machine Learning-generated Building and Road Dataset: The Case of Rwanda

Nyandwi,

Gerke,

Achanccaray

2024

PFG

View full text Add to dashboard Cite

Accurate and up-to-date building and road data are crucial for informed spatial planning. In developing regions in particular, major challenges arise due to the limited availability of these data, primarily as a result of the inherent inefficiency of traditional field-based surveys and manual data generation methods. Importantly, this limitation has prompted the exploration of alternative solutions, including the use of remote sensing machine learning-generated (RSML) datasets. Within the field of RSML datasets, a plethora of models have been proposed. However, these methods, evaluated in a research setting, may not translate perfectly to massive real-world applications, attributable to potential inaccuracies in unknown geographic spaces. The scepticism surrounding the usefulness of datasets generated by global models, owing to unguaranteed local accuracy, appears to be particularly concerning. As a consequence, rigorous evaluations of these datasets in local scenarios are essential for gaining insights into their usability. To address this concern, this study investigates the local accuracy of large RSML datasets. For this evaluation, we employed a dataset generated using models pre-trained on a variety of samples drawn from across the world and accessible from public repositories of open benchmark datasets. Subsequently, these models were fine-tuned with a limited set of local samples specific to Rwanda. In addition, the evaluation included Microsoft’s and Google’s global datasets. Using ResNet and Mask R‑CNN, we explored the performance variations of different building detection approaches: bottom-up, end-to-end, and their combination. For road extraction, we explored the approach of training multiple models on subsets representing different road types. Our testing dataset was carefully designed to be diverse, incorporating both easy and challenging scenes. It includes areas purposefully chosen for their high level of clutter, making it difficult to detect structures like buildings. This inclusion of complex scenarios alongside simpler ones allows us to thoroughly assess the robustness of DL-based detection models for handling diverse real-world conditions. In addition, buildings were evaluated using a polygon-wise comparison, while roads were assessed using network length-derived metrics.Our results showed a precision (P) of around 75% and a recall (R) of around 60% for the locally fine-tuned building model. This performance was achieved in three out of six testing sites and is considered the lowest limit needed for practical utility of RSML datasets, according to the literature. In contrast, comparable results were obtained in only one out of six sites for the Google and Microsoft datasets. Our locally fine-tuned road model achieved moderate success, meeting the minimum usability threshold in four out of six sites. In contrast, the Microsoft dataset performed well on all sites. In summary, our findings suggest improved performance in road extraction, relative to building extraction tasks. Moreover, we observed that a pipeline relying on a combination of bottom-up and top-down segmentation, while leveraging open global benchmark annotation dataset as well as a small number of samples for fine-tuning, can offer more accurate RSML datasets compared to an open global dataset. Our findings suggest that relying solely on aggregated accuracy metrics can be misleading. According to our evaluation, even city-level derived measures may not capture significant variations in performance within a city, such as lower accuracy in specific neighbourhoods. Overcoming the challenges of complex areas might benefit from exploring alternative approaches, including the integration of LiDAR data, UAV images, aerial images or using other network architectures.

show abstract

An enhanced N-point interpolation method to eliminate average precision distortion

Cited by 4 publications

References 5 publications

Key Indicators to Assess the Performance of LiDAR-Based Perception Algorithms: A Literature Review

Key Indicators to Assess the Performance of LiDAR-Based Perception Algorithms: A Literature Review

An Enhanced YOLOv4 Model With Self-Dependent Attentive Fusion and Component Randomized Mosaic Augmentation for Metal Surface Defect Detection

Local Evaluation of Large-scale Remote Sensing Machine Learning-generated Building and Road Dataset: The Case of Rwanda

Contact Info

Product

Resources

About