Enabling Massive XML-Based Biological Data Management in HBase

Liu, Jian; Liu, Qiuru; Zhang, Lei; Su, Shuhui; Liu, Yongzhuang

doi:10.1109/tcbb.2019.2915811

Cited by 12 publications

(6 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As shown in Figure 2 , for the submission process, the hyper-converged architecture integrates the server, storage, and network resources into a virtual pool, so that it can achieve high scalability using a distributed scale-out expansion for storage and computing resources. Reanalyzed datasets, such as that for proteins, peptides and spectra, are stored in a distributed column-oriented storage database known as HBase ( 13 ) in the Hadoop cluster. The indexes of the identifications are stored in an Elastic Search cluster for a distributed, high-scalable, real-time search and data analysis.…”

Section: Big Data Architecture and Infrastructure Of Iproxmentioning

confidence: 99%

iProX in 2021: connecting proteomics data sharing with big data

Chen

Liu

et al. 2021

Nucleic Acids Research

335

169

View full text Add to dashboard Cite

The rapid development of proteomics studies has resulted in large volumes of experimental data. The emergence of big data platform provides the opportunity to handle these large amounts of data. The integrated proteome resource, iProX (https://www.iprox.cn), which was initiated in 2017, has been greatly improved with an up-to-date big data platform implemented in 2021. Here, we describe the main iProX developments since its first publication in Nucleic Acids Research in 2019. First, a hyper-converged architecture with high scalability supports the submission process. A hadoop cluster can store large amounts of proteomics datasets, and a distributed, RESTful-styled Elastic Search engine can query millions of records within one second. Also, several new features, including the Universal Spectrum Identifier (USI) mechanism proposed by ProteomeXchange, RESTful Web Service API, and a high-efficiency reanalysis pipeline, have been added to iProX for better open data sharing. By the end of August 2021, 1526 datasets had been submitted to iProX, reaching a total data volume of 92.42TB. With the implementation of the big data platform, iProX can support PB-level data storage, hundreds of billions of spectra records, and second-level latency service capabilities that meet the requirements of the fast growing field of proteomics.

show abstract

Section: Big Data Architecture and Infrastructure Of Iproxmentioning

confidence: 99%

iProX in 2021: connecting proteomics data sharing with big data

Chen

Liu

et al. 2021

Nucleic Acids Research

335

169

View full text Add to dashboard Cite

show abstract

“…[ 9,10 ] Cloud providers such as Amazon Web Services (AWS), [ 11 ] Microsoft Azure, [ 12 ] and Google Cloud Platform offer a range of services and tools that can be used to store, process, and analyze precision medicine data. [ 10,13 ] NoSQL Databases [ 14 ] : NoSQL databases such as MongoDB, [ 15–17 ] Cassandra, [ 15,18 ] and HBase [ 19 ] are used to store and manage large‐scale genomic and clinical datasets. These databases offer scalability, flexibility, and high availability, making them well‐suited to handle the complex and dynamic nature of precision medicine data.…”

Section: Datawarehouse and Data Managementmentioning

confidence: 99%

Biomedical Big Data Technologies, Applications, and Challenges for Precision Medicine: A Review

Yang,

Huang,

Yang

et al. 2023

Global Challenges

View full text Add to dashboard Cite

The explosive growth of biomedical Big Data presents both significant opportunities and challenges in the realm of knowledge discovery and translational applications within precision medicine. Efficient management, analysis, and interpretation of big data can pave the way for groundbreaking advancements in precision medicine. However, the unprecedented strides in the automated collection of large‐scale molecular and clinical data have also introduced formidable challenges in terms of data analysis and interpretation, necessitating the development of novel computational approaches. Some potential challenges include the curse of dimensionality, data heterogeneity, missing data, class imbalance, and scalability issues. This overview article focuses on the recent progress and breakthroughs in the application of big data within precision medicine. Key aspects are summarized, including content, data sources, technologies, tools, challenges, and existing gaps. Nine fields—Datawarehouse and data management, electronic medical record, biomedical imaging informatics, Artificial intelligence‐aided surgical design and surgery optimization, omics data, health monitoring data, knowledge graph, public health informatics, and security and privacy—are discussed.

show abstract

“…In order to effectively manage massive biomedical data, approaches concerned with the reengineering traditional databases in HBase are greatly needed. The adoption of the HBase naturally triggers the requirement of the mapping from the historical one to the new one [43], [44]. Although HBase is employed to model the future data explosion, little work focuses on the uncertainty modeling of objectoriented biomedical information in HBase.…”

Section: Introductionmentioning

confidence: 99%

Uncertainty Modeling of Object-Oriented Biomedical Information in HBase

Zhang

Sun

et al. 2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

While object-oriented databases (OODBs) are known to be rich in functionality, HBase database, which is a distributed and scalable big data store, as well as uncertain databases have recently gained a lot of attention in the database community. This paper presents a methodology for handling an important step of knowledge integrations and migrations. In particular, a formal approach for reengineering fuzzy object-oriented databases in HBase is firstly developed. The reengineering approach is based on the technique of rule-based schema mapping, which defines a set of transformation rules involved in the process of schema transformations for mapping a fuzzy object-oriented database schema into a fuzzy HBase database schema. In addition, a formal approach to map the fuzzy object-oriented algebra into fuzzy HBase algebra is proposed. On this basis, we complement the work with a comprehensive set of experiments to show the efficiency of our proposed approach in terms of query time and scalability metrics. INDEX TERMS Big data store, fuzzy HBase, fuzzy object-oriented database, mapping.

show abstract

Enabling Massive XML-Based Biological Data Management in HBase

Cited by 12 publications

References 38 publications

iProX in 2021: connecting proteomics data sharing with big data

iProX in 2021: connecting proteomics data sharing with big data

Biomedical Big Data Technologies, Applications, and Challenges for Precision Medicine: A Review

Uncertainty Modeling of Object-Oriented Biomedical Information in HBase

Contact Info

Product

Resources

About