Crowdsourcing image analysis for plant phenomics to generate ground truth data for machine learning

Zhou, Naihui; Siegel, Zachary D.; Zarecor, Scott; Lee, Nigel; Campbell, Darwin A.; Andorf, Carson M.; Nettleton, Dan; Lawrence‐Dill, Carolyn J.; Ganapathysubramanian, Baskar; Kelly, Jonathan W.; Friedberg, Iddo

doi:10.1371/journal.pcbi.1006337

Cited by 52 publications

(38 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this context, the concept of “ground truth,” which means checking the results of machine learning for accuracy against the real world, is fundamental for validating AI performance. In a radiology context, this might mean confirming diagnoses suggested by AI by comparison to pathological or surgical diagnoses; ground truth is the data assumed to be true [11]. Machine learning has been likened to training a dog: “reinforcing good behaviour, ignoring bad, and giving her enough practice to work out what to do for herself” [12].…”

Section: Definitionsmentioning

confidence: 99%

What the radiologist should know about artificial intelligence – an ESR white paper

Neri¹,

Souza²,

Brady³

et al. 2019

Insights Imaging

219

121

View full text Add to dashboard Cite

This paper aims to provide a review of the basis for application of AI in radiology, to discuss the immediate ethical and professional impact in radiology, and to consider possible future evolution. Even if AI does add significant value to image interpretation, there are implications outside the traditional radiology activities of lesion detection and characterisation. In radiomics, AI can foster the analysis of the features and help in the correlation with other omics data. Imaging biobanks would become a necessary infrastructure to organise and share the image data from which AI models can be trained. AI can be used as an optimising tool to assist the technologist and radiologist in choosing a personalised patient’s protocol, tracking the patient’s dose parameters, providing an estimate of the radiation risks. AI can also aid the reporting workflow and help the linking between words, images, and quantitative data. Finally, AI coupled with CDS can improve the decision process and thereby optimise clinical and radiological workflow.

show abstract

Section: Definitionsmentioning

confidence: 99%

What the radiologist should know about artificial intelligence – an ESR white paper

Neri¹,

Souza²,

Brady³

et al. 2019

Insights Imaging

219

121

View full text Add to dashboard Cite

show abstract

“…However, the power of AI/machine learning is limited by the quality and quantity of ground-truth data. This presents an ongoing opportunity for citizen scientists to provide data that improve the utility of the technology itself, as with citizen science approaches regarding ground-truthing [55].…”

Section: Emerging Opportunities and Challengesmentioning

confidence: 99%

The role of citizen science in addressing grand challenges in food and agriculture research

et al. 2018

View full text Add to dashboard Cite

The power of citizen science to contribute to both science and society is gaining increased recognition, particularly in physics and biology. Although there is a long history of public engagement in agriculture and food science, the term ‘citizen science’ has rarely been applied to these efforts. Similarly, in the emerging field of citizen science, most new citizen science projects do not focus on food or agriculture. Here, we convened thought leaders from a broad range of fields related to citizen science, agriculture, and food science to highlight key opportunities for bridging these overlapping yet disconnected communities/fields and identify ways to leverage their respective strengths. Specifically, we show that (i) citizen science projects are addressing many grand challenges facing our food systems, as outlined by the United States National Institute of Food and Agriculture, as well as broader Sustainable Development Goals set by the United Nations Development Programme, (ii) there exist emerging opportunities and unique challenges for citizen science in agriculture/food research, and (iii) the greatest opportunities for the development of citizen science projects in agriculture and food science will be gained by using the existing infrastructure and tools of Extension programmes and through the engagement of urban communities. Further, we argue there is no better time to foster greater collaboration between these fields given the trend of shrinking Extension programmes, the increasing need to apply innovative solutions to address rising demands on agricultural systems, and the exponential growth of the field of citizen science.

show abstract

“…Even if the feature is unfamiliar to most people, crowdsourcing may be viable if the task is simple and the feature obvious. In a recent study on best practices for crowdsourcing plant feature annotation, Zhou et al (2018) found that, with minimal instruction, anonymous online workers could accurately identify maize male flowers in images where they were clearly visible. Accurate identification of many plant features requires a certain level of expertise, however.…”

Section: Introductionmentioning

confidence: 99%

Millimeter-Level Plant Disease Detection From Aerial Photographs via Deep Learning and Crowdsourced Data

Wiesner-Hanks

Stewart

et al. 2019

Front. Plant Sci.

View full text Add to dashboard Cite

Computer vision models that can recognize plant diseases in the field would be valuable tools for disease management and resistance breeding. Generating enough data to train these models is difficult, however, since only trained experts can accurately identify symptoms. In this study, we describe and implement a two-step method for generating a large amount of high-quality training data with minimal expert input. First, experts located symptoms of northern leaf blight (NLB) in field images taken by unmanned aerial vehicles (UAVs), annotating them quickly at low resolution. Second, non-experts were asked to draw polygons around the identified diseased areas, producing high-resolution ground truths that were automatically screened based on agreement between multiple workers. We then used these crowdsourced data to train a convolutional neural network (CNN), feeding the output into a conditional random field (CRF) to segment images into lesion and non-lesion regions with accuracy of 0.9979 and F1 score of 0.7153. The CNN trained on crowdsourced data showed greatly improved spatial resolution compared to one trained on expert-generated data, despite using only one fifth as many expert annotations. The final model was able to accurately delineate lesions down to the millimeter level from UAV-collected images, the finest scale of aerial plant disease detection achieved to date. The two-step approach to generating training data is a promising method to streamline deep learning approaches for plant disease detection, and for complex plant phenotyping tasks in general.

show abstract

Crowdsourcing image analysis for plant phenomics to generate ground truth data for machine learning

Cited by 52 publications

References 26 publications

What the radiologist should know about artificial intelligence – an ESR white paper

What the radiologist should know about artificial intelligence – an ESR white paper

The role of citizen science in addressing grand challenges in food and agriculture research

Millimeter-Level Plant Disease Detection From Aerial Photographs via Deep Learning and Crowdsourced Data

Contact Info

Product

Resources

About