BackgroundThis article provides an overview of the first BioASQ challenge, a competition on large-scale biomedical semantic indexing and question answering (QA), which took place between March and September 2013. BioASQ assesses the ability of systems to semantically index very large numbers of biomedical scientific articles, and to return concise and user-understandable answers to given natural language questions by combining information from biomedical articles and ontologies.ResultsThe 2013 BioASQ competition comprised two tasks, Task 1a and Task 1b. In Task 1a participants were asked to automatically annotate new PubMed documents with MeSH headings. Twelve teams participated in Task 1a, with a total of 46 system runs submitted, and one of the teams performing consistently better than the MTI indexer used by NLM to suggest MeSH headings to curators. Task 1b used benchmark datasets containing 29 development and 282 test English questions, along with gold standard (reference) answers, prepared by a team of biomedical experts from around Europe and participants had to automatically produce answers. Three teams participated in Task 1b, with 11 system runs. The BioASQ infrastructure, including benchmark datasets, evaluation mechanisms, and the results of the participants and baseline methods, is publicly available.ConclusionsA publicly available evaluation infrastructure for biomedical semantic indexing and QA has been developed, which includes benchmark datasets, and can be used to evaluate systems that: assign MeSH headings to published articles or to English questions; retrieve relevant RDF triples from ontologies, relevant articles and snippets from PubMed Central; produce “exact” and paragraph-sized “ideal” answers (summaries). The results of the systems that participated in the 2013 BioASQ competition are promising. In Task 1a one of the systems performed consistently better from the NLM’s MTI indexer. In Task 1b the systems received high scores in the manual evaluation of the “ideal” answers; hence, they produced high quality summaries as answers. Overall, BioASQ helped obtain a unified view of how techniques from text classification, semantic indexing, document and passage retrieval, question answering, and text summarization can be combined to allow biomedical experts to obtain concise, user-understandable answers to questions reflecting their real information needs.Electronic supplementary materialThe online version of this article (doi:10.1186/s12859-015-0564-6) contains supplementary material, which is available to authorized users.

No abstract

No abstract

No abstract

No abstract

No abstract

EU Directive 86/609/EEC for the protection of laboratory animals obliges scientists to consider whether a planned animal experiment can be replaced, reduced or refined (3Rs principle). To meet this regulatory obligation, scientists must consult the relevant scientific literature prior to any experimental study using laboratory animals. More than 50 million potentially 3Rs relevant documents are spread over the World Wide Web, biomedical literature and patent databases. In April 2008, the beta version of Go3R ("www.Go3R.org":http://www.Go3R.org), the first knowledge-based semantic search engine for alternative methods to animal experiments, was released. Go3R is free of charge and enables scientists and regulatory authorities involved in the planning, authorisation and performance of animal experiments to determine the availability of alternative methods in a fast and comprehensive manner. The technical basis of this search engine is specific 3Rs expert knowledge captured within the Go3R Ontology containing 87,218 labels and synonyms. A total of 16,620 concepts were structured in 28 branches, where 1,227 concepts were newly defined to specifically describe directly 3Rs relevant knowledge. Additionally relevant headings from MeSH where referenced to reflect the topics associated with the definition of Animal Testing Alternatives. Therefore it is distinguished between thematic-defining and directly 3Rs relevant branches. In addition to the assignment of direct parent-child relationships, further relationship types were introduced to allow to model 3Rs relevant domain knowledge. Examples for such knowledge are e.g. (1) the characteristics of cell culture tests methods, which usually utilize “specific cell types” or “cell lines” and are associated with a specific “endpoint” and “endpoint detection method” or (2) named test methods like “PREDISAFE™”, which replaces an animal test namely the “eye irritation test” in rabbits and uses specific cells namely “SIRC Cells” or (3) the “Haemagglutinin-Neuraminidase Protein Assay”, which detects a protein of the “Newcastle disease virus”. Thereby, an article in which e.g. a specific 3Rs method is not explicitly mentioned could still be recognized as relevant for the specific topic searched for in an indirect manner, for example if it mentions specific cells, endpoints or endpoint detection methods, which are relevant for the respective application. The search engine Go3R with its novel ontology is already well recognized by the 3Rs community and will be further maintained and developed.

No abstract