Evaluation of Relational and NoSQL Approaches for Cohort Identification from Heterogeneous Data Sources in the National Sleep Research Resource

Abstract

Patient cohort identification across heterogeneous data sources is a challenging task, which may involve a complicated process of data loading, harmonization and querying. Most existing cohort identification tools use a relational database model implemented in SQL for storing patient data. However, SQL databases have restrictions on the maximum number of columns in a table, which necessitates the breaking down of high dimensional data into multiple tables and as a consequence affects query performance. In this paper, we developed two NoSQL-based patient cohort query systems based on an existing SQL-based system for the cross-cohort query in the National Sleep Resource Research (NSRR). We used eight NSRR datasets in our experiment to evaluate the performance of the NoSQLbased and SQL-based systems in data loading, harmonization and query. Our experiment showed that NoSQL-based approaches outperformed the SQL-based and are rather promising for developing patient cohort query systems across heterogeneous data sources.

GENE2D: A NoSQL Integrated Data Repository of Genetic Disorders Data

“…Two NoSQL-based patient cohort query systems are used in comparison to a SQL-based system to evaluate their performance in supporting high-dimensional and heterogeneous data sources. The NoSQL databases exceeded the maximum limits for the number of table columns in traditional relational databases and successfully integrated eight datasets into NoSQL databases [34]. Driven by NoSQL document database features such as being opensource and cost-effective, with a flexible schema model, indexing capabilities, and scalability for future expansion, we decided to be among the first to use the NoSQL document store as a back-end for an IDR for genetic data, to the best of our knowledge.…”