On expert curation and scalability: UniProtKB/Swiss-Prot as a case study

Abstract

MotivationBiological knowledgebases, such as UniProtKB/Swiss-Prot, constitute an essential component of daily scientific research by offering distilled, summarized and computable knowledge extracted from the literature by expert curators. While knowledgebases play an increasingly important role in the scientific community, their ability to keep up with the growth of biomedical literature is under scrutiny. Using UniProtKB/Swiss-Prot as a case study, we address this concern via multiple literature triage approaches.ResultsWith the assistance of the PubTator text-mining tool, we tagged more than 10 000 articles to assess the ratio of papers relevant for curation. We first show that curators read and evaluate many more papers than they curate, and that measuring the number of curated publications is insufficient to provide a complete picture as demonstrated by the fact that 8000–10 000 papers are curated in UniProt each year while curators evaluate 50 000–70 000 papers per year. We show that 90% of the papers in PubMed are out of the scope of UniProt, that a maximum of 2–3% of the papers indexed in PubMed each year are relevant for UniProt curation, and that, despite appearances, expert curation in UniProt is scalable.Availability and implementationUniProt is freely available at http://www.uniprot.org/.Supplementary information Supplementary data are available at Bioinformatics online.

UPCLASS: a Deep Learning-based Classifier for UniProtKB Entry Publications

“…While for some database resources it has been shown that expert curation can keep up with the exponential growth of the scientific literature [4], scaling-up biocuration remains a challenge. A key success factor for UniProtKB's scalability, for example, is that the set of expert curated literature in the knowledge base focus on non-redundant annotations for proteins.…”

“…The reviewed section represents less than 1% of the knowledgebase. Biocurators select a subset of the available literature for a given protein, representing the landscape of knowledge at a given time [4].…”

“…To capture the breadth of publications about proteins and make it easily available to users, UniProt compiles additional bibliography from three types of external sourcesdatabases, community and text mining -which complements the curated literature set with additional publications and adds relevant literature to entries not yet curated [4] [23]. UniProtKB publications in reviewed entries are categorised on eleven predefined topics -Expression, Family & Domains, Function, Interaction, Names, Pathology & Biotech, PTM / Processing, Sequences, Structure, Subcellular Location, and Miscellaneous -based on the annotation they contribute to the protein entry, e.g., a paper that is the evidence source for the protein catalytic activity will be included in the category Function (Figure 1).…”

Funding knowledgebases: Towards a sustainable funding model for the UniProt use case

“…This income hardly allows the curation of the current relevant literature 5 . However, it does not allow any expansion that is required by the fast-growing need of literature biocuration.…”

“…Coherently with this analysis, a recent paper demonstrated that the costs of curation are quite modest on a per-article basis, and represent a fraction of the cost of the original research: the cost of biocuration of articles for the EcoCyc database is estimated at $ 219 (€193) per article over a 5-year period, corresponding to 6—15% of the cost of open-access publication fees for publishing biomedical articles, and to 0.088% of the cost of the overall research project associated 4 . Additionally, a recent analysis on UniProt showed that expert annotation is sustainable given that a large part of the literature is redundant and/or not relevant for the curation 5 . Thus, curation costs are affordable in an absolute sense and represent a small fraction of the cost of the overall associated research projects that generated the experimental data.…”