Informatics Tools for Cancer Research and Care: Bridging the Gap Between Innovation and Implementation

Warner, Jeremy L.; Klemm, Juli

doi:10.1200/cci.20.00086

Cited by 1 publication

(1 citation statement)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Limited time (68% of respondents) and funding (57% respondents) were major barriers for performing software impact evaluations (respondents could select multiple barriers). Although a few funding mechanisms support the maintenance and analysis of software (as opposed to creation of new software), such as the ITCR sustainment awards ( Kibbe et al 2017 , Warner and Klemm 2020 ), or the Essential Open Source Software for Science program of the Chan Zuckerberg Initiative ( Science 2019 ), more funding for software sustainability is needed compared to what is currently available. Awareness of this need was also demonstrated by the recent Declaration on Funding Research Software Sustainability by the Research Software Alliance (ReSA) ( Barker et al 2023 ).…”

Section: Methodsmentioning

confidence: 99%

Best practices to evaluate the impact of biomedical research software—metric collection beyond citations

Afiaz,

Ivanov,

Chamberlin

et al. 2024

Bioinformatics

View full text Add to dashboard Cite

Motivation Software is vital for the advancement of biology and medicine. Impact evaluations of scientific software have primarily emphasized traditional citation metrics of associated papers, despite these metrics inadequately capturing the dynamic picture of impact and despite challenges with improper citation. Results To understand how software developers evaluate their tools, we conducted a survey of participants in the Informatics Technology for Cancer Research (ITCR) program funded by the National Cancer Institute (NCI). We found that although developers realize the value of more extensive metric collection, they find a lack of funding and time hindering. We also investigated software among this community for how often infrastructure that supports more nontraditional metrics were implemented and how this impacted rates of papers describing usage of the software. We found that infrastructure such as social media presence, more in-depth documentation, the presence of software health metrics, and clear information on how to contact developers seemed to be associated with increased mention rates. Analyzing more diverse metrics can enable developers to better understand user engagement, justify continued funding, identify novel use cases, pinpoint improvement areas, and ultimately amplify their software’s impact. Challenges are associated, including distorted or misleading metrics, as well as ethical and security concerns. More attention to nuances involved in capturing impact across the spectrum of biomedical software is needed. For funders and developers, we outline guidance based on experience from our community. By considering how we evaluate software, we can empower developers to create tools that more effectively accelerate biological and medical research progress. Availability and Implementation More information about the analysis, as well as access to data and code is available at https://github.com/fhdsl/ITCR_Metrics_manuscript_website. Supplementary Information Supplementary Information are available at Bioinformatics online.

show abstract

Section: Methodsmentioning

confidence: 99%