Framework for Supporting Genomic Operations

Kaitoua, Abdulrahman; Pinoli, Pietro; Bertoni, Michele; Ceri, Stefano

doi:10.1109/tc.2016.2603980

Cited by 22 publications

(38 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We expect all clauses to be well formed, possibly because the clause DLE(Max) is automatically added at the end of the string, where Max is a problem-specific maximum distance. As discussed in [23], the DLE and UP/DOWN clauses are commutative with the other clauses and can be pushed at the beginning of the string, thus delimiting the search space. After clause rewriting, the clauses appearing in the predicate before the MD clause -if present -are denoted as first-step clauses; they include a DLE clause and may include a DGE and a UP/DOWN clause.…”

Section: Joinmentioning

confidence: 99%

“…The join implementation strategy, discussed in [23], occurs by first considering the distal conditions of the first-step clauses (as defined in Section 2.2.1); this part of the computation is the most expensive as it requires computing all the matching pairs, while the subsequent computation, when present, selects some of them into the result. Therefore, binning optimization is concerned with the search space as defined by the first-step clauses.…”

Section: Optimal Binning For Joinmentioning

confidence: 99%

“…Compared with join, the map operation is simpler; the algorithm used to compare reference and experiment regions in each bin, described in [23], uses merge-sort, whose complexity, as a function of the bin size, can be approximated by:…”

Section: Optimal Binning For Mapmentioning

confidence: 99%

“…In a previous paper [23] we presented our framework for the parallel implementation of GMQL operations on the cloud and the algorithms for the implementation of domain-specific GMQL operations using Flink [8] and Spark [9]. These algorithms use binning, i.e.…”

Section: Introductionmentioning

confidence: 99%

“…In order to make this paper self-contained, Section 2 summarizes the GMQL data model, the JOIN and MAP operations, and the strategy for binning the genome as a function of the specific JOIN and MAP conditions. Together with the data model and operations we also introduce the definition and evaluation of their profiles, while the detailed implementation of JOIN and MAP operations using Flink and Spark is omitted, and can be found on [23]. Section 3 then presents the analytical models for computing the optimal bin size of JOIN and MAP, and Section 4 provides the validation of the models both on synthetic and real datasets, with a sensitivity analysis showing the dependency of the optimal choice of binning from the query parameters.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Optimal Binning for Genomics

Gulino

Kaitoua²,

Ceri

2019

IEEE Trans. Comput.

Self Cite

View full text Add to dashboard Cite

Genome sequencing is expected to be the most prolific source of big data in the next decade; millions of whole genome datasets will open new opportunities for biological research and personalized medicine. Genome sequences are abstracted in the form of interesting regions, describing abnormalities of the genome. The parallel execution on the cloud of complex operations for joining and mapping billions of genomic regions is increasingly important. Genome binning, i.e. partitioning of the genome into small-size segments, adapts classic data partitioning methods to genomics; region distributions to bins must reflect operation-specific correctness rules. As a consequence, determining the optimal bin size for such operations is a complex mathematical problem, whose solution requires careful modeling. The main result of this paper is the mathematical formulation and solution of the optimal binning problem for join and map operations in the context of GMQL, a query language over genomic regions; the model is validated by experiments showing its accuracy and sensitivity to the variation of operations' parameters. We also optimize sequences of operations by inheriting the binning between two consecutive operations and we show the deployment of GMQL and the tuning of the proposed model on different cloud computing systems.

show abstract

Section: Joinmentioning

confidence: 99%

Section: Optimal Binning For Joinmentioning

confidence: 99%

Section: Optimal Binning For Mapmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Optimal Binning for Genomics

Gulino

Kaitoua²,

Ceri

2019

IEEE Trans. Comput.

Self Cite

View full text Add to dashboard Cite

show abstract

Implementing a Transcription Factor Interaction Prediction System Using the GenoMetric Query Language

Perna

Canakoglu

Pinoli

et al. 2018

Methods in Molecular Biology

View full text Add to dashboard Cite

Novel technologies and growing interest have resulted in a large increase in the amount of data available for genomics and transcriptomics studies, both in terms of volume and contents. Biology is relying more and more on computational methods to process, investigate, and extract knowledge from this huge amount of data. In this work, we present the TICA web server (available at http://www.gmql.eu/tica/ ), a fast and compact tool developed to support data-driven knowledge discovery in the realm of transcription factor interaction prediction. TICA leverages both the GenoMetric Query Language, a novel query tool (based on the Apache Hadoop and Spark technologies) specialized in the integration and management of heterogeneous, large genomic datasets, and a statistical method for robust detection of co-locations across interval-based data, in order to infer physically interacting transcription factors. Notably, TICA allows investigators to upload and analyze their own ChIP-seq experiments datasets, comparing them both against ENCODE data or between themselves, achieving computation time which increases linearly with respect to dataset size and density. Using ENCODE data from three well-studied cell lines as reference, we show that TICA predictions are supported by existing biological knowledge, making the web server a reliable and efficient tool for interaction screening and data-driven hypothesis generation.

show abstract

Evaluating Genomic Big Data Operations on SciDB and Spark

Cattani

Ceri

Kaitoua

et al. 2017

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

We are developing a new, holistic data management system for genomics, which provides high-level abstractions for querying large genomic datasets. We designed our system so that it leverages on data management engines for low-level data access. Such design can be adapted to two different kinds of data engines: the family of scientific databases (among them, SciDB) and the broader family of generic platforms (among them, Spark). Trade-offs are not obvious; scientific databases are expected to outperform generic platforms when they use features which are embedded within their specialized design, but generic platforms are expected to outperform scientific databases on generalpurpose operations.In this paper, we compare our SciDB and Spark implementations at work on genomic abstractions. We use four typical genomic operations as benchmark, stemming from the concrete requirements of our project, and encoded using SciDB and Spark; we discuss their common aspects and differences, specifically discussing how genomic regions and operations can be expressed using SciDB arrays. We comparatively evaluate the performance and scalability of the two implementations over datasets consisting of billions of genomic regions.

show abstract

Framework for Supporting Genomic Operations

Cited by 22 publications

References 37 publications

Optimal Binning for Genomics

Optimal Binning for Genomics

Implementing a Transcription Factor Interaction Prediction System Using the GenoMetric Query Language

Evaluating Genomic Big Data Operations on SciDB and Spark

Contact Info

Product

Resources

About