Quanti.us: a tool for rapid, flexible, crowd-based annotation of images

Hughes, Alex J.; Mornin, Joseph D.; Biswas, Subhendu; Beck, Lauren E.; Bauer, Diana; Raj, Arjun; Bianco, Simone; Gartner, Zev J.

doi:10.1038/s41592-018-0069-0

Cited by 55 publications

(41 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Alternative methods for collecting image annotations now exist, such as the Quanti.us system (30) for distributing manual annotations to nonexpert workers on the internet. Alternative methods for collecting image annotations now exist, such as the Quanti.us system (30) for distributing manual annotations to nonexpert workers on the internet.…”

Section: Expert Annotationsmentioning

confidence: 99%

“…Our prototype tool was useful to collect nucleus annotations for this research; however, significant development is needed to improve it. Alternative methods for collecting image annotations now exist, such as the Quanti.us system (30) for distributing manual annotations to nonexpert workers on the internet. This may enable the annotation process for new projects to be scaled up quickly to many more images, and, according to their findings, reaching similar precision to experts when multiple workers provide independent annotation replicates.…”

Section: Expert Annotationsmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of Deep Learning Strategies for Nucleus Segmentation in Fluorescence Images

et al. 2019

View full text Add to dashboard Cite

Identifying nuclei is often a critical first step in analyzing microscopy images of cells and classical image processing algorithms are most commonly used for this task. Recent developments in deep learning can yield superior accuracy, but typical evaluation metrics for nucleus segmentation do not satisfactorily capture error modes that are relevant in cellular images. We present an evaluation framework to measure accuracy, types of errors, and computational efficiency; and use it to compare deep learning strategies and classical approaches. We publicly release a set of 23,165 manually annotated nuclei and source code to reproduce experiments and run the proposed evaluation methodology. Our evaluation framework shows that deep learning improves accuracy and can reduce the number of biologically relevant errors by half. © 2019 The Authors. Cytometry Part A published by Wiley Periodicals, Inc. on behalf of International Society for Advancement of Cytometry.

show abstract

Section: Expert Annotationsmentioning

confidence: 99%

Section: Expert Annotationsmentioning

confidence: 99%

Evaluation of Deep Learning Strategies for Nucleus Segmentation in Fluorescence Images

et al. 2019

View full text Add to dashboard Cite

show abstract

“…For applications that require much greater scale, recruitment of more annotators may be necessary. Several methods for recruiting annotators include paying annotators a small amount of money for each annotation by integrating with Amazon Turk [9], gamifying the app [23], or creating a citizen science effort [1]. We envision that this type of accurately human-curated images will support (by providing ground truth) and complement (in rare and unanticipated scenarios)machine-learning approaches as they become dominant in image-based analyses for many fields of scientific inquiry.…”

Section: Future Perspectivementioning

confidence: 99%

“…Some automated tools require extensive tuning or parameter optimization prior to annotation to enhance accuracy, and many image processing pipelines are not well-suited for heterogeneous image sets. In addition, many tools for human annotation limit the way users can define image features of interest, for example, via rectangles, polygonsor circles [9]. Annotation speed is limited by the complexity of annotation softwareand, ultimately, how quickly annotators can mark phenotypes accurately [10].…”

Section: Introductionmentioning

confidence: 99%

Fast, Versatile and Quantitative Annotation of Complex Images

et al. 2019

View full text Add to dashboard Cite

We report a generic smartphone app for quantitative annotation of complex images. The app is simple enough to be used by children, and annotation tasks are distributed across app users, contributing to efficient annotation. We demonstrate its flexibility and speed by annotating >30,000 images, including features of rice root growth and structure, stem cell aggregate morphology, and complex worm (Caenorhabditis elegans) postures, for which we show that the speed of annotation is >130-fold faster than state-of-the-art techniques with similar accuracy.

show abstract

“…hampering efforts to apply deep learning in many biological domains 4 , we opted to use a pragmatic approach, treating the segmentation and classification steps as separate problems (Fig. 1a).…”

mentioning

confidence: 99%

A fully automated deep learning pipeline for high-throughput colony segmentation and classification

Carl

Duempelmann

Shimada

et al. 2019

Preprint

View full text Add to dashboard Cite

Background: Adenine auxotrophy is a commonly used non-selective genetic marker in yeast research.It allows investigators to easily visualize and quantify various genetic and epigenetic events by simply reading out colony color. However, manual counting of large numbers of colonies is extremely timeconsuming, difficult to reproduce and possibly inaccurate.Results: Using cutting-edge neural networks, we have developed a fully automated pipeline for colony segmentation and classification, which speeds up white/red colony quantification 100-fold over manual counting by an experienced researcher. Conclusions: Our approach uses readily available training data and can be smoothly integrated intoexisting protocols, vastly speeding up screening assays and increasing the statistical power of experiments that employ adenine auxotrophy. KeywordsDeep learning, neural networks, adenine auxotrophy, yeast, growth assay BackgroundAuxotrophy is the inability of an organism to synthesize a particular organic compound required for its growth. For example, yeasts with mutations in the adenine biosynthetic pathway cannot grow on media lacking adenine. When grown under adenine-limiting conditions, adenine auxotrophs grow but accumulate a cell-limited red pigment in their vacuoles, whereas wild type cells grow white. This red/white colony color assay has been widely used over the last decades for the investigation of many biological processes, such as recombination, copy number, chromosome loss, plasmid stability, prion propagation, or epigenetic gene regulation in both budding and fission yeasts. However, adapting this assay to quantitative high-throughput applications has proven challenging, as it requires extensive scoring of colony phenotypes by eye. In addition to being time-consuming and tedious, manual colony scoring may suffer from inaccuracy and irreproducibility. Nonetheless, up to now manual scoring is a common practice in the yeast community. Modern machine-learning techniques such as deep learning have made huge strides in automated image classification in recent years and are beginning to be applied to previously intractable problems in the biomedical imaging domain. We set out to leverage these recent developments to build a computational pipeline that would enable fully automated high-throughput adenine auxotrophy-based screening and quantification.Typically, red/white colony color assays start with plating individual yeast cells on limiting adenine indicator plates and allowing them to grow until they form colonies large enough to be inspected by eye.Each plate may represent an independent condition or genotype, the penetrance of which can be assessed by quantifying the percentage of non-white colonies per plate. In order to create a pipeline that would fit into existing protocols as seamlessly as possible, we considered our input to be images of plates and our desired output to be percentages of white and non-white colonies per plate. Two major tasks are required to generate this output: separating the colonies from th...

show abstract

Quanti.us: a tool for rapid, flexible, crowd-based annotation of images

Cited by 55 publications

References 39 publications

Evaluation of Deep Learning Strategies for Nucleus Segmentation in Fluorescence Images

Evaluation of Deep Learning Strategies for Nucleus Segmentation in Fluorescence Images

Fast, Versatile and Quantitative Annotation of Complex Images

A fully automated deep learning pipeline for high-throughput colony segmentation and classification

Contact Info

Product

Resources

About