Key points

No abstract

No abstract

No abstract

ObjectiveTo compare expert assessment with bibliometric indicators as tools to assess the quality and importance of scientific research papers.Methods and MaterialsShortly after their publication in 2005, the quality and importance of a cohort of nearly 700 Wellcome Trust (WT) associated research papers were assessed by expert reviewers; each paper was reviewed by two WT expert reviewers. After 3 years, we compared this initial assessment with other measures of paper impact.ResultsShortly after publication, 62 (9%) of the 687 research papers were determined to describe at least a ‘major addition to knowledge’ –6 were thought to be ‘landmark’ papers. At an aggregate level, after 3 years, there was a strong positive association between expert assessment and impact as measured by number of citations and F1000 rating. However, there were some important exceptions indicating that bibliometric measures may not be sufficient in isolation as measures of research quality and importance, and especially not for assessing single papers or small groups of research publications.ConclusionWhen attempting to assess the quality and importance of research papers, we found that sole reliance on bibliometric indicators would have led us to miss papers containing important results as judged by expert review. In particular, some papers that were highly rated by experts were not highly cited during the first three years after publication. Tools that link expert peer reviews of research paper quality and importance to more quantitative indicators, such as citation analysis would be valuable additions to the field of research assessment and evaluation.

More evidence of the meaning and validity of ALMs and altmetrics, coupled with greater consistency and transparency in their presentation, would enable research funders to explore their potential value and identify appropriate use cases.

Background: CatSper1 and CatSper2 are two recently identified channel-like proteins, which show sperm specific expression patterns. Through targeted mutagenesis in the mouse, CatSper1 has been shown to be required for fertility, sperm motility and for cAMP induced Ca 2+ current in sperm. Both channels resemble a single pore forming repeat from a four repeat voltage dependent Ca 2+ /Na + channel. However, neither CatSper1 or CatSper2 have been shown to function as cation channels when transfected into cells, singly or in conjunction. As the pore forming units of voltage gated cation channels form a tetramer it has been suggested that the known CatSper proteins require additional subunits and/or interaction partners to function.

ObjectivesTo investigate the feasibility of using research papers cited in clinical guidelines as a way to track the impact of particular funding streams or sources.SettingIn recent years, medical research funders have made efforts to enhance the understanding of the impact of their funded research and to provide evidence of the ‘value’ of investments in particular areas of research. One of the most challenging areas of research evaluation is around impact on policy and practice. In the UK, the National Institute of Health and Clinical Excellence (NICE) provide clinical guidelines, which bring together current high-quality evidence on the diagnosis and treatment of clinical problems. Research referenced in these guidelines is an indication of its potential to have real impact on health policy and practice.DesignThis study is based on analysis of the authorship and funding attribution of research cited in two NICE clinical guidelines: dementia and chronic obstructive pulmonary disease.ResultsAnalysis identified that around a third of papers cited in the two NICE guidelines had at least one author based in the UK. In both cases, about half of these UK attributed papers contained acknowledgements which allowed the source of funding for the research to be identified. The research cited in these guidelines was found to have been supported by a diverse set of funders from different sectors. The study also investigated the contribution of research groups based in universities, industry and the public sector.ConclusionsThe study found that there is great potential for guidelines to be used as sources of information on the quality of the research used in their development and that it is possible to track the source of the funding of the research. The challenge is in harnessing the relevant information to track this in an efficient way.