No abstract

No abstract

Purpose: Computed-based skill assessment relies on accurate metrics to provide comprehensive feedback to trainees. Improving the accuracy of video-based metrics computed using object detection is generally done by improving the performance of the object detection network, however increasing its performance requires resources that cannot always be obtained. This study aims to improve the accuracy of metrics in central venous catheterization without requiring a high performing object detection network by removing false positive predictions identified using uncertainty quantification. Methods: The uncertainty for each bounding box was calculated using an entropy equation. The uncertainties were then compared to an uncertainty threshold computed using the optimal point of a Receiver Operating Characteristic curve. Predictions were removed if the uncertainty fell below the predefined threshold. 50 videos were recorded and annotated with ground truth bounding boxes. These bounding boxes were used to train an object detection network, which was used to produce predictive bounding boxes for the test set. This method was evaluated by computing metrics for the predictive bounding boxes with and without having false positives removed and comparing them to ground truth labels using a Pearson Correlation. Results: The Pearson Correlations for the baseline comparisons and the comparisons made using the results calculated using false positive removal were 0.922 and 0.816 for syringe path lengths, 0.753 and 0.510 for ultrasound path lengths, 0.831 and 0.489 for ultrasound usage times, and 0.857 and 0.805 for syringe usage times. Conclusions: This method consistently reduced inflated metrics, making it promising for improving metric accuracy.

No abstract