Towards reproducible, transparent, and systematic benchmarking of omics computational tools

Abstract

Rapid technological advances, such as the development of next-generation sequencing, are driving the need for comprehensive computational tools to analyze the wealth of generated genomic data. Systematic benchmarking has been successful in many research disciplines to help non-computational researchers evaluate the accuracy and applicability of new computational tools. Adopting a standardized benchmarking practice and following established principles for the design of new benchmarking studies could help researchers using omics data to better leverage technological innovation. In this Review, we propose step-by-step instructions for increasing reusability, transparency, and reproducibility of benchmarking studies by using containerization, common data representation, open data, and systematic parameter description.