Quantifying the Effects of Ground Truth Annotation Quality on Object Detection and Instance Segmentation Performance

Agnew, Cathaoir; Eising, Ciaran; Denny, Patrick; Scanlan, Anthony; Ven, Pepijn van de; Grua, Eoin Martino

doi:10.1109/access.2023.3256723

Cited by 3 publications

(2 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Two fundamentally different HTR models that are both considered state of the art in HTR, are used for handwriting recognition. Py-Laia [19] is a more traditional CRNN, with a stack of convolutions, 2 This only applies to the end of line similar to VGG [23], followed by a bidirectional LSTM [11] and a Connectionist Temporal Classification (CTC) [10] for the decoding part. TrOCR decided to combine a pre-trained Vision Transformer (ViT) [7] with a pre-trained language model, such as BERT [6], and since they are sharing the Transformer architecture, they can easily be combined into one model.…”

Section: Handwriting Recognitionmentioning

confidence: 99%

“…In related fields, several studies have been conducted to investigate the impact of ground truth quality on deep learning, for example in the context of object detection [2,13], text-line segmentation [3,22], and semantic segmentation [20,25] in natural images or historical document images. However, the problems encountered for HTR are specific and to the best of our knowledge, there are currently no comprehensive studies on the impact of ground-truth quality for deep learning-based HTR.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Impact of the ground truth quality for handwriting recognition

Jungo,

Vögtlin,

Fakhari

et al. 2023

Proceedings of the 12th International Symposium on Information and Communication Technology

View full text Add to dashboard Cite

Handwriting recognition is a key technology for accessing the content of old manuscripts, helping to preserve cultural heritage. Deep learning shows an impressive performance in solving this task. However, to achieve its full potential, it requires a large amount of labeled data, which is difficult to obtain for ancient languages and scripts. Often, a trade-off has to be made between ground truth quantity and quality, as is the case for the recently introduced Bullinger database. It contains an impressive amount of over a hundred thousand labeled text line images of mostly premodern German and Latin texts that were obtained by automatically aligning existing page-level transcriptions with text line images. However, the alignment process introduces systematic errors, such as wrongly hyphenated words. In this paper, we investigate the impact of such errors on training and evaluation and suggest means to detect and correct typical alignment errors.

show abstract

Section: Handwriting Recognitionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Impact of the ground truth quality for handwriting recognition

Jungo,

Vögtlin,

Fakhari

et al. 2023

Proceedings of the 12th International Symposium on Information and Communication Technology

View full text Add to dashboard Cite

show abstract

Real-time canola damage detection: An end-to-end framework with semi-automatic crusher and lightweight ShuffleNetV2_YOLOv5s

Thakuria,

Erkinbaev

2024

Smart Agricultural Technology

View full text Add to dashboard Cite

The METRIC-framework for assessing data quality for trustworthy AI in medicine: a systematic review

Schwabe,

Becker,

Seyferth

et al. 2024

npj Digit. Med.

View full text Add to dashboard Cite

The adoption of machine learning (ML) and, more specifically, deep learning (DL) applications into all major areas of our lives is underway. The development of trustworthy AI is especially important in medicine due to the large implications for patients’ lives. While trustworthiness concerns various aspects including ethical, transparency and safety requirements, we focus on the importance of data quality (training/test) in DL. Since data quality dictates the behaviour of ML products, evaluating data quality will play a key part in the regulatory approval of medical ML products. We perform a systematic review following PRISMA guidelines using the databases Web of Science, PubMed and ACM Digital Library. We identify 5408 studies, out of which 120 records fulfil our eligibility criteria. From this literature, we synthesise the existing knowledge on data quality frameworks and combine it with the perspective of ML applications in medicine. As a result, we propose the METRIC-framework, a specialised data quality framework for medical training data comprising 15 awareness dimensions, along which developers of medical ML applications should investigate the content of a dataset. This knowledge helps to reduce biases as a major source of unfairness, increase robustness, facilitate interpretability and thus lays the foundation for trustworthy AI in medicine. The METRIC-framework may serve as a base for systematically assessing training datasets, establishing reference datasets, and designing test datasets which has the potential to accelerate the approval of medical ML products.

show abstract

Quantifying the Effects of Ground Truth Annotation Quality on Object Detection and Instance Segmentation Performance

Cited by 3 publications

References 33 publications

Impact of the ground truth quality for handwriting recognition

Impact of the ground truth quality for handwriting recognition

Real-time canola damage detection: An end-to-end framework with semi-automatic crusher and lightweight ShuffleNetV2_YOLOv5s

The METRIC-framework for assessing data quality for trustworthy AI in medicine: a systematic review

Contact Info

Product

Resources

About