Data Science for Genomic Data Management: Challenges, Resources, Experiences

Abstract

We highlight several challenges which are faced by data scientists who use public datasets for solving biological and clinical problems. In spite of the large efforts in building such public datasets, they are dispersed over many sources and heterogeneous for their formats and sequencing/calling techniques, often meeting highly variable quality standards. Moreover, for most research questions, scientists hardly find datasets with enough samples for building and training machine learning models. Data scarcity depends on the complexity of the genomic domain, with its multi-facets, as well as the lack of organic initiatives to provide standardization across communities. In this paper, we discuss our approach to genomic data management, that can strongly improve the problems of data dispersion and format heterogeneity through high-level abstractions for genomics. We briefly present the computational resources that were recently developed by the GeCo project (ERC Advanced Grant); they include GDM, a Genomic Data Model providing interoperability across data formats; GMQL, a genometric query language for answering data science queries over genomic datasets; and an in-house integrated repository providing attribute-based and keyword-based search over normalized metadata from several open data repositories. We describe these resources at work on a specific research question, and we highlight how we managed to produce a model for addressing such specific research question by overcoming the lack of sufficient samples and labelled datasets.

Teaching Mathematics to Computer Scientists: Reflections and a Case Study

“…For example, the programming of 3D graphics and animation in games requires the implementation of mathematical equations [17,27]. There is a degree of presence of mathematics in various computer science tasks such as cybersecurity [5,40], artificial intelligence [18,43] and data science [11].…”

SparkGC: Spark based genome compression for large collections of genomes

“…Another problem is that with the continuous increase of genomic data, cloud storage, the storage mode specially designed for big data, has gradually entered the bioinformatics community [ 8 ]. Cloud storage enables users to use storage facilities on demand without the huge cost of building and maintaining expensive infrastructure.…”