Data Lakes, Clouds, and Commons: A Review of Platforms for Analyzing and Sharing Genomic Data

Grossman, Robert L.

doi:10.1016/j.tig.2018.12.006

Cited by 47 publications

(34 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Author Manuscript Published OnlineFirst on February 12, 2020; DOI: 10.1158/1055-9965.EPI-19-0842 infrastructure meets the CRDC's requirements (22,23). This demonstrates that the DW infrastructure SDSC offers is consistent with the research community's current standards and reaffirms that DWs have the flexibility and scalability to support CEC research.…”

Section: Discussionmentioning

confidence: 59%

Insights from Adopting a Data Commons Approach for Large-scale Observational Cohort Studies: The California Teachers Study

Lacey

Chung

Hughes

et al. 2020

Cancer Epidemiology, Biomarkers &Amp; Prevention

View full text Add to dashboard Cite

words; limit is 250)Background: Large-scale cancer epidemiology cohorts (CECs) have successfully collected, analyzed, and shared patient-reported data for years. CECs increasingly need to make their data more Findable, Accessible, Interoperable, and Reusable, or FAIR. How CECs should approach this transformation is unclear. Methods:The California Teachers Study (CTS) is an observational CEC of 133,477 participants followed since 1995-1996. In 2014, we began updating our data storage, management, analysis, and sharing strategy. With the San Diego Supercomputer Center, we deployed a new infrastructure based on a Data Warehouse, to integrate and manage data; and a secure and shared workspace with documentation, software, and analytic tools that facilitate collaboration and accelerate analyses. Results: Our new CTS infrastructure includes a Data Warehouse and data marts, which are focused subsets from the Data Warehouse designed for efficiency. The secure CTS workspace utilizes a Remote Desktop service that operates within a HIPAA and FISMA compliant platform. Our infrastructure offers broad access to CTS data; includes statistical analysis and data visualization software and tools; flexibly managesother key data activities (e.g., cleaning, updates, & data sharing); and will continue to evolve to advance FAIR principles.on July 15, 2020. Conclusion:Our scalable infrastructure provides the security, authorization, data model, metadata, and analytic tools needed to manage, share, and analyze CTS data in ways that are consistent with the NCI's CancerResearch Data Commons Framework. Impact:The CTS's implementation of new infrastructure in an ongoing CEC demonstrates how population sciences can explore and embrace new cloud-based and analytics infrastructure to accelerate cancer research and translation.on July 15, 2020.

show abstract

Section: Discussionmentioning

confidence: 59%

Insights from Adopting a Data Commons Approach for Large-scale Observational Cohort Studies: The California Teachers Study

Lacey

Chung

Hughes

et al. 2020

Cancer Epidemiology, Biomarkers &Amp; Prevention

View full text Add to dashboard Cite

show abstract

“…Recent progress has brought substantial transformations in how the petabytes of genomic data being generated each year are assimilated and analysed, including the emergence of cloud-based and federated approaches. Effective and efficient management of increasingly complex genomic datasets requires addressing challenges with these emerging approaches as well as innovations in the use of hardware, algorithms, software, standards, and platforms 40 . Current barriers include the lack of interoperable genomic data resources (which limits downstream access, integration, and analyses) and the absence of controlled and consistently adopted data and metadata vocabularies and ontologies 41,42 .…”

Section: Boxmentioning

confidence: 99%

Strategic vision for improving human health at The Forefront of Genomics

Green

Gunter

Biesecker

et al. 2020

Nature

221

178

View full text Add to dashboard Cite

“…If participants' or collaborators' institutions are equipped with large in-house high-performance computing resources, they will likely have more direct access and practical assistance in their genome project. Otherwise, cloud-based computing is a potential solution that has been widely emphasized in previous works including easy-to-follow steps [88][89][90]. While cloud computing provides flexibility, competitive pricing, and continually updated hardware and software, it still requires assistance from information technology (IT) specialists to set up suitable cloud-based software.…”

Section: Step 6: Check the Computational Resources And Requirementsmentioning

confidence: 99%

Twelve quick steps for genome assembly and annotation in the classroom

et al. 2020

View full text Add to dashboard Cite

Eukaryotic genome sequencing and de novo assembly, once the exclusive domain of well-funded international consortia, have become increasingly affordable, thus fitting the budgets of individual research groups. Third-generation long-read DNA sequencing technologies are increasingly used, providing extensive genomic toolkits that were once reserved for a few select model organisms. Generating high-quality genome assemblies and annotations for many aquatic species still presents significant challenges due to their large genome sizes, complexity, and high chromosome numbers. Indeed, selecting the most appropriate sequencing and software platforms and annotation pipelines for a new genome project can be daunting because tools often only work in limited contexts. In genomics, generating a high-quality genome assembly/annotation has become an indispensable tool for better understanding the biology of any species. Herein, we state 12 steps to help researchers get started in genome projects by presenting guidelines that are broadly applicable (to any species), sustainable over time, and cover all aspects of genome assembly and annotation projects from start to finish. We review some commonly used approaches, including practical methods to extract high-quality DNA and choices for the best sequencing platforms and library preparations. In addition, we discuss the range of potential bioinformatics pipelines, including structural and functional annotations (e.g., transposable elements and repetitive sequences). This paper also includes information on how to build a wide community for a genome project, the importance of data management, and how to make the data and results Findable, Accessible, Interoperable, and Reusable (FAIR) by submitting them to a public repository and sharing them with the research community.

show abstract

Data Lakes, Clouds, and Commons: A Review of Platforms for Analyzing and Sharing Genomic Data

Cited by 47 publications

References 61 publications

Insights from Adopting a Data Commons Approach for Large-scale Observational Cohort Studies: The California Teachers Study

Insights from Adopting a Data Commons Approach for Large-scale Observational Cohort Studies: The California Teachers Study

Strategic vision for improving human health at The Forefront of Genomics

Twelve quick steps for genome assembly and annotation in the classroom

Contact Info

Product

Resources

About