Human-in-the-loop machine learning with applications for population health

Chen, Long; Wang, Jiangtao; Guo, Bin; Chen, Liming

doi:10.1007/s42486-022-00115-4

Cited by 5 publications

(1 citation statement)

References 34 publications

(33 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Critical to the success of this pipeline has been the recognition that community science tools can be deployed at multiple stages across the overall workflow (e.g., humans in the loop; Chen et al, 2023 ) to help either scale up production of training data (critical for the label finder effort) or for data correction and improvement (both for finding and classifying labels and for OCR). This approach best uses human effort but requires thought and care in assuring that tasks are simple enough while still being engaging, and that documentation is as clear as possible to cover any corner cases or other challenges.…”

Section: Discussionmentioning

confidence: 99%

Humans in the loop: Community science and machine learning synergies for overcoming herbarium digitization bottlenecks

Guralnick,

LaFrance,

Denslow

et al. 2024

Appl Plant Sci

View full text Add to dashboard Cite

PremiseAmong the slowest steps in the digitization of natural history collections is converting imaged labels into digital text. We present here a working solution to overcome this long‐recognized efficiency bottleneck that leverages synergies between community science efforts and machine learning approaches.MethodsWe present two new semi‐automated services. The first detects and classifies typewritten, handwritten, or mixed labels from herbarium sheets. The second uses a workflow tuned for specimen labels to label text using optical character recognition (OCR). The label finder and classifier was built via humans‐in‐the‐loop processes that utilize the community science Notes from Nature platform to develop training and validation data sets to feed into a machine learning pipeline.ResultsOur results showcase a >93% success rate for finding and classifying main labels. The OCR pipeline optimizes pre‐processing, multiple OCR engines, and post‐processing steps, including an alignment approach borrowed from molecular systematics. This pipeline yields >4‐fold reductions in errors compared to off‐the‐shelf open‐source solutions. The OCR workflow also allows human validation using a custom Notes from Nature tool.DiscussionOur work showcases a usable set of tools for herbarium digitization including a custom‐built web application that is freely accessible. Further work to better integrate these services into existing toolkits can support broad community use.

show abstract

Section: Discussionmentioning

confidence: 99%

Humans in the loop: Community science and machine learning synergies for overcoming herbarium digitization bottlenecks

Guralnick,

LaFrance,

Denslow

et al. 2024

Appl Plant Sci

View full text Add to dashboard Cite

show abstract

Empowering Zero-Shot Object Detection: A Human-in-the-Loop Strategy for Unveiling Unseen Realms in Visual Data

Kerdvibulvech,

2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Addressing the data bottleneck in medical deep learning models using a human-in-the-loop machine learning approach

Mosqueira-Rey,

Hernández-Pereira,

Bobes-Bascarán

et al. 2023

Neural Comput & Applic

View full text Add to dashboard Cite

Any machine learning (ML) model is highly dependent on the data it uses for learning, and this is even more important in the case of deep learning models. The problem is a data bottleneck, i.e. the difficulty in obtaining an adequate number of cases and quality data. Another issue is improving the learning process, which can be done by actively introducing experts into the learning loop, in what is known as human-in-the-loop (HITL) ML. We describe an ML model based on a neural network in which HITL techniques were used to resolve the data bottleneck problem for the treatment of pancreatic cancer. We first augmented the dataset using synthetic cases created by a generative adversarial network. We then launched an active learning (AL) process involving human experts as oracles to label both new cases and cases by the network found to be suspect. This AL process was carried out simultaneously with an interactive ML process in which feedback was obtained from humans in order to develop better synthetic cases for each iteration of training. We discuss the challenges involved in including humans in the learning process, especially in relation to human–computer interaction, which is acquiring great importance in building ML models and can condition the success of a HITL approach. This paper also discusses the methodological approach adopted to address these challenges.

show abstract

Human-in-the-loop machine learning with applications for population health

Cited by 5 publications

References 34 publications

Humans in the loop: Community science and machine learning synergies for overcoming herbarium digitization bottlenecks

Humans in the loop: Community science and machine learning synergies for overcoming herbarium digitization bottlenecks

Empowering Zero-Shot Object Detection: A Human-in-the-Loop Strategy for Unveiling Unseen Realms in Visual Data

Addressing the data bottleneck in medical deep learning models using a human-in-the-loop machine learning approach

Contact Info

Product

Resources

About