Crowdsourcing biocuration: The Community Assessment of Community Annotation with Ontologies (CACAO)

Abstract

Experimental data about gene functions curated from the primary literature have enormous value for research scientists in understanding biology. Using the Gene Ontology (GO), manual curation by experts has provided an important resource for studying gene function, especially within model organisms. Unprecedented expansion of the scientific literature and validation of the predicted proteins have increased both data value and the challenges of keeping pace. Capturing literature-based functional annotations is limited by the ability of biocurators to handle the massive and rapidly growing scientific literature. Within the community-oriented wiki framework for GO annotation called the Gene Ontology Normal Usage Tracking System (GONUTS), we describe an approach to expand biocuration through crowdsourcing with undergraduates. This multiplies the number of high-quality annotations in international databases, enriches our coverage of the literature on normal gene function, and pushes the field in new directions. From an intercollegiate competition judged by experienced biocurators, Community Assessment of Community Annotation with Ontologies (CACAO), we have contributed nearly 5,000 literature-based annotations. Many of those annotations are to organisms not currently well-represented within GO. Over a 10-year history, our community contributors have spurred changes to the ontology not traditionally covered by professional biocurators. The CACAO principle of relying on community members to participate in and shape the future of biocuration in GO is a powerful and scalable model used to promote the scientific enterprise. It also provides undergraduate students with a unique and enriching introduction to critical reading of primary literature and acquisition of marketable skills.

Student biocuration projects as a learning environment

“…Peer review of the annotations by the students themselves would provide a more efficient biocuration project model, and this has been achieved, by a group at Texas A&M, through the establishment of an annotation competition. 12 As significant resources are needed to train new biocurators before they are able to work independently, 29 biocuration projects, such as the ones provided at UCL, provide substantial initial training for students as biocurators, Indeed, three of our respondents are currently working as biocurators after completion of their projects.…”

“…However, several students were motivated to volunteer to participate in future curation projects because of the satisfaction of knowing that their annotations were being shared and would be of benefit to the wider scientific community. Unfortunately, there are only a very few examples of successful community biocuration efforts 8 , 10 , 12 , 41 , 42 and, therefore, the opportunities for the wider scientific community to contribute to this need are limited. One of the rate limiting factors in community curation is checking the accuracy of the submitted annotations by expert biocurators.…”

“…The CACAO competition model has addressed this issue by ensuring annotations are first peer reviewed by other competitors, which reduces the number of errors that need to be corrected before the data is incorporated into a public database. 12 As well as benefiting the individual, through continued professional development, community biocuration would result in an enriched coverage of the literature benefiting the wider research community. Another potential source of community curators is retired scientists.…”

“…There is now accumulating evidence that student biocuration projects not only promote better understanding of molecular and cellular biology but can also contribute valuable high-quality annotations that improve existing resources. 9 - 12 All of the respondents would recommend this project to future students, regardless of study stage or intended future career with respondent 3445942 stating: “I would highly recommend bioinformatics-based projects for students. It’s an area that is growing tremendously leading to many different career paths in life sciences.…”

“…While many of these courses focus on providing training on specific resources, such as Ensembl, 3 UniProt 4 or functional analysis tools, 5 - 7 a few universities, including University College London (UCL), encourage students to contribute annotations to public biological knowledgebases. 8 - 12 For example, over the past 10 years the Functional Gene Annotation group (UCL) has provided a 10-week bioinformatics MSc module as well as a Gene Ontology biocuration project to undergraduate and postgraduate students.…”