Learning from multiple annotators for medical image segmentation

Zhang, Le; Tanno, Ryutaro; Xu, Mou-Cheng; Huang, Yawen; Bronik, Kevin; Chen, Jin; Jacob, Joseph; Zheng, Yefeng; Shao, Ling; Ciccarelli, Olga; Barkhof, Frederik; Alexander, Daniel C.

doi:10.1016/j.patcog.2023.109400

Cited by 13 publications

(16 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ML models can be less subjective compared to manual image labeling [46]. It is possible for a single person or group of people to classify many images, in which case the ML model trained on these data would likely capture subtle biases or errors from the sole person annotating the training data.…”

Section: Manual Image Labelingmentioning

confidence: 99%

“…While the 9-class model accuracies (78.1% for RF model and 73.1% for DNN model) may be less than desired, it is important to remember that ML models can only be as good as the data and labels used to train them [46]. Generating the training dataset for the models here required coordinating multiple domain experts, and the time required to classify the entire dataset and reconcile classification labels was substantial.…”

Section: Future Workmentioning

confidence: 99%

See 1 more Smart Citation

Classification of Lakebed Geologic Substrate in Autonomously Collected Benthic Imagery Using Machine Learning

Geisz,

Wernette,

Esselman

2024

Remote Sensing

View full text Add to dashboard Cite

Mapping benthic habitats with bathymetric, acoustic, and spectral data requires georeferenced ground-truth information about habitat types and characteristics. New technologies like autonomous underwater vehicles (AUVs) collect tens of thousands of images per mission making image-based ground truthing particularly attractive. Two types of machine learning (ML) models, random forest (RF) and deep neural network (DNN), were tested to determine whether ML models could serve as an accurate substitute for manual classification of AUV images for substrate type interpretation. RF models were trained to predict substrate class as a function of texture, edge, and intensity metrics (i.e., features) calculated for each image. Models were tested using a manually classified image dataset with 9-, 6-, and 2-class schemes based on the Coastal and Marine Ecological Classification Standard (CMECS). Results suggest that both RF and DNN models achieve comparable accuracies, with the 9-class models being least accurate (~73–78%) and the 2-class models being the most accurate (~95–96%). However, the DNN models were more efficient to train and apply because they did not require feature estimation before training or classification. Integrating ML models into benthic habitat mapping process can improve our ability to efficiently and accurately ground-truth large areas of benthic habitat using AUV or similar images.

show abstract

Section: Manual Image Labelingmentioning

confidence: 99%

Section: Future Workmentioning

confidence: 99%

Classification of Lakebed Geologic Substrate in Autonomously Collected Benthic Imagery Using Machine Learning

Geisz,

Wernette,

Esselman

2024

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…The many manual labels can help train algorithms that account for idiosyncratic fluorescence and noise distributions within each image dataset but then necessitate labels for each imaging condition. Generating such labels is time consuming and subject to human error ( Giovannucci et al, 2019 ; Zhang et al, 2020 ).…”

Section: Introductionmentioning

confidence: 99%

A Semi-supervised Pipeline for Accurate Neuron Segmentation with Fewer Ground Truth Labels

Baker,

Gong

2024

eNeuro

View full text Add to dashboard Cite

Recent advancements in two-photon calcium imaging have enabled scientists to record the activity of thousands of neurons with cellular resolution. This scope of data collection is crucial to understanding the next generation of neuroscience questions, but analyzing these large recordings requires automated methods for neuron segmentation. Supervised methods for neuron segmentation achieve state of-the-art-accuracy and speed, but currently require large amounts of manually generated ground truth training labels. We reduced the required number of training labels by designing a semi-supervised pipeline. Our pipeline used neural network ensembling to generate pseudolabels to train a single shallow U-Net. We tested our method on three publicly available datasets and compared our performance to three widely-used segmentation methods. Our method outperformed other methods when trained on a small number of ground truth labels and could achieve state-of-the-art accuracy after training on approximately a quarter of the number of ground truth labels as supervised methods. When trained on many ground truth labels, our pipeline attained higher accuracy than that of state-of-the-art methods. Overall, our work will help researchers accurately process large neural recordings while minimizing the time and effort needed to generate manual labels.Significance statementModern neuroscience analyzes the activity of hundreds to thousands of neurons from large optical imaging datasets. One important step in this analysis is neuron segmentation. Supervised algorithms have performed neuron segmentation with class-leading accuracy and speed but lag unsupervised algorithms in training time. A large component of training time is the manual labeling of neurons as training samples; current supervised methods train over many manual labels to achieve accurate prediction. We developed a semi-supervised neuron segmentation algorithm, SAND, that retained high accuracy in the few-label regime. SAND employed neural network ensembling to generate robust pseudolabels and used domain-specific hyperparameter optimization. SAND was more accurate than existing supervised and unsupervised algorithms in low and high label regimes of multiple imaging conditions.

show abstract

“…Majority of the previous studies used manual segmentation; however, it is time-consuming, tedious, and prone to human errors (30)(31)(32), and a semi-automatic method using MRI data resampling, image filtering, and region-growing was adopted to segment the PG efficiently (33,34). However, even a semi-automatic segmentation is a daunting task due to several factors (26): 1) The small size and complex shape of the PG; 2) the anatomical and pathological variations among individuals; 3) the low contrast between different tissues; and 4) the lack of standardized criteria or protocols for segmentation.…”

Section: Introductionmentioning

confidence: 99%

deepPGSegNet: MRI-based pituitary gland segmentation using deep learning

Choi,

Sung,

Ogawa

2024

Front. Endocrinol.

View full text Add to dashboard Cite

IntroductionIn clinical research on pituitary disorders, pituitary gland (PG) segmentation plays a pivotal role, which impacts the diagnosis and treatment of conditions such as endocrine dysfunctions and visual impairments. Manual segmentation, which is the traditional method, is tedious and susceptible to inter-observer differences. Thus, this study introduces an automated solution, utilizing deep learning, for PG segmentation from magnetic resonance imaging (MRI).MethodsA total of 153 university students were enrolled, and their MRI images were used to build a training dataset and ground truth data through manual segmentation of the PGs. A model was trained employing data augmentation and a three-dimensional U-Net architecture with a five-fold cross-validation. A predefined field of view was applied to highlight the PG region to optimize memory usage. The model’s performance was tested on an independent dataset. The model’s performance was tested on an independent dataset for evaluating accuracy, precision, recall, and an F1 score.Results and discussionThe model achieved a training accuracy, precision, recall, and an F1 score of 92.7%, 0.87, 0.91, and 0.89, respectively. Moreover, the study explored the relationship between PG morphology and age using the model. The results indicated a significant association between PG volume and midsagittal area with age. These findings suggest that a precise volumetric PG analysis through an automated segmentation can greatly enhance diagnostic accuracy and surveillance of pituitary disorders.

show abstract

Learning from multiple annotators for medical image segmentation

Cited by 13 publications

References 25 publications

Classification of Lakebed Geologic Substrate in Autonomously Collected Benthic Imagery Using Machine Learning

Classification of Lakebed Geologic Substrate in Autonomously Collected Benthic Imagery Using Machine Learning

A Semi-supervised Pipeline for Accurate Neuron Segmentation with Fewer Ground Truth Labels

deepPGSegNet: MRI-based pituitary gland segmentation using deep learning

Contact Info

Product

Resources

About