OnTimeURB: Multi-Cloud Resource Brokering for Bioinformatics Workflows

Pandey, Ashish; Lyu, Zhen; Joshi, Trupti; Calyam, Prasad

doi:10.1109/bibm47256.2019.8983386

Cited by 9 publications

(11 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They can also utilize analysis and visualization tools such as Venn diagram, volcano plot, hierarchical clustering heatmap, and PCA analysis. KBCommons also provides users access to automated workflows to analyze the next‐generation, high‐throughput sequencing data, using the Pegasus Workflow Management System (Pegasus WMS) and OnTimeURB capabilities (Pandey et al, 2019) for customizing their analysis. Several predesigned bioinformatics workflows have already been integrated into KBCommons, such as the RNA‐Seq analysis workflow, PGen (Liu et al, 2016), which is used to analyze genomic variations in the workflow, such as copy number variations, methylation, and single cell RNA‐Seq analysis.…”

Section: Bioinformatics Methods and Tools For Multiomics Datamentioning

confidence: 99%

Bioinformatics for plant and agricultural discoveries in the age of multiomics: A review and case study of maize nodal root growth under water deficit

Wang

Sen

Zeng

et al. 2022

Physiologia Plantarum

Self Cite

View full text Add to dashboard Cite

Advances in next‐generation sequencing and other high‐throughput technologies have facilitated multiomics research, such as genomics, epigenomics, transcriptomics, proteomics, metabolomics, and phenomics. The resultant emerging multiomics data have brought new challenges as well as opportunities, as seen in the plant and agriculture science domains. We reviewed several bioinformatic and computational methods, models, and platforms, and we have highlighted some of our in‐house developed efforts aimed at multiomics data analysis, integration, and management issues faced by the research community. A case study using multiomics datasets generated from our studies of maize nodal root growth under water deficit stress demonstrates the power of these datasets and some other publicly available tools. This analysis also sheds light on the landscape of such applied bioinformatic tools currently available for plant and crop science studies and introduces emerging trends and how they may affect the future.

show abstract

Section: Bioinformatics Methods and Tools For Multiomics Datamentioning

confidence: 99%

Bioinformatics for plant and agricultural discoveries in the age of multiomics: A review and case study of maize nodal root growth under water deficit

Wang

Sen

Zeng

et al. 2022

Physiologia Plantarum

Self Cite

View full text Add to dashboard Cite

show abstract

“…This Cloud Solution Template Recommender uses k-Nearest Neighbors (KNN) Machine learning algorithm and Integer Linear Programming (ILP) to recommend a suitable cloud template based on user requirements 32,33 (both functional and nonfunctional requirements).It features a Component Abstraction Model to implement intelligent resource "abstractions" coupled with "reusable" hardware and software configurations in the form of "custom templates" to simplify heterogeneous resource management efforts. In this recommender, an offline initial recommender module improves the user productivity by narrowing down cloud resource compositions to the most appropriate options.…”

Section: Cloud Solution Template Recommendermentioning

confidence: 99%

“…In this section, we present evaluation experiments using the Cloud Solution Template Recommender 33 based on user requirements, in comparison to the state-of-the-art k-NN approach. 32 For experimental simulation, three bioninformatics application workflows were selected, namely Fastqc (small-scale resources), RNASeq (medium-scale resources), and PGen (high-scale resources).…”

Section: Evaluation Of Recommenders In the Kbcommons Science Gatewaymentioning

confidence: 99%

“…In this section, we present evaluation experiments using the Cloud Solution Template Recommender 33 in the KBCommons bioinformatics science gateway. Our experiments show the Cloud Solution Template Recommender's consistency in composing cost‐effective templates solutions based on user requirements, in comparison to the state‐of‐the‐art k‐NN approach 32 .…”

Section: Evaluation Studiesmentioning

confidence: 99%

See 1 more Smart Citation

Recommender‐as‐a‐service with chatbot guided domain‐science knowledge discovery in a science gateway

Vekaria

Calyam

Sivarathri

et al. 2020

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

Scientists in disciplines such as neuroscience and bioinformatics are increasingly relying on science gateways for experimentation on voluminous data, as well as analysis and visualization in multiple perspectives. Though current science gateways provide easy access to computing resources, data sets and tools specific to the disciplines, scientists often use slow and tedious manual efforts to perform knowledge discovery to accomplish their research/education tasks. Recommender systems can provide expert guidance and can help them to navigate and discover relevant publications, tools, data sets, or even automate cloud resource configurations suitable for a given scientific task.To realize the potential of integration of recommenders in science gateways in order to spur research productivity, we present a novel "OnTimeRecommend" recommender system. The OnTimeRecommend comprises of several integrated recommender modules implemented as microservices that can be augmented to a science gateway in the form of a recommender-as-a-service. The guidance for use of the recommender modules in a science gateway is aided by a chatbot plug-in viz., Vidura Advisor. To validate our OnTimeRecommend, we integrate and show benefits for both novice and expert users in domain-specific knowledge discovery within two exemplar science gateways, one in neuroscience (CyNeuro) and the other in bioinformatics (KBCommons). K E Y W O R D Schatbot-guided user interface, knowledge discovery, microservices, recommender system, science gateway INTRODUCTIONRecent science and engineering research tasks are increasingly becoming data-intensive and thus relying on workflows to automate integration and analysis of voluminous data to test hypotheses. For example, research and training in neural science and engineering increasingly deal with diverse and voluminous multiparameter data, 1 posing unique challenges outlined in an NSF iNeuro report 2 as limited access to: multisomics data archives, 3 heterogeneous software 4 and computing resources (Neuroscience Gateway, 5 Amazon Web Services [AWS]), and multisite interdisciplinary expertise (e.g., engineering, biology and psychology). Existing distributed high-performance computing resources (HPC) and other cyberinfrastructure (CI) tools for data management support the related data analysis and visualization capabilities. However, to fully utilize such capabilities, neuroscientists (often with limited CI skills) are required to take valuable time away from the focus of knowledge discovery in neuroscience, in order to learn about how to use the various technologies.

show abstract

“…To mitigate the problem of ill-advised resource allocation certain key factors can be identified that strongly govern the selection of optimal cloud resources for maximum resource usage and user satisfaction. These factors are performance, agility, cost, and security offered by the CSPs [20]. Since selection and configuration of multi-cloud resources for modern applications requires handling objective factors such as performance, agility, cost, and security (i.e., PACS factors), the multi-cloud resource brokering involves a multi-dimensional optimization problem in resource selection.…”

Section: Overviewmentioning

confidence: 99%

Trusted resource allocation in volunteer edge-cloud computing for scientific applications

Pandey¹

View full text Add to dashboard Cite

Data-intensive science applications in fields such as e.g., bioinformatics, health sciences, and material discovery are becoming increasingly dynamic and demanding with resource requirements. Researchers using these applications which are based on advanced scientific workflows frequently require a diverse set of resources that are often not available within private servers or a single Cloud Service Provider (CSP). For example, a user working with Precision Medicine applications would prefer only those CSPs who follow guidelines from HIPAA (Health Insurance Portability and Accountability Act) for implementing their data services and might want services from other CSPs for economic viability. With the generation of more and more data these workflows often require deployment and dynamic scaling of multi-cloud resources in an efficient and high-performance manner (e.g., quick setup, reduced computation time, and increased application throughput). At the same time, users seek to minimize the costs of configuring the related multi-cloud resources. While performance and cost are among the key factors to decide upon CSP resource selection, the scientific workflows often process proprietary/confidential data that introduces additional constraints of security postures. Thus, users have to make an informed decision on the selection of resources that are most suited for their applications while trading off between the key factors of resource selection which are performance, agility, cost, and security (PACS). Furthermore, even with the most efficient resource allocation across multi-cloud, the cost to solution might not be economical for all users which have led to the development of new paradigms of computing such as volunteer computing where users utilize volunteered cyber resources to meet their computing requirements. For economical and readily available resources, it is essential that such volunteered resources can integrate well with cloud resources for providing the most efficient computing infrastructure for users. In this dissertation, individual stages such as user requirement collection, user's resource preferences, resource brokering and task scheduling, in lifecycle of resource brokering for users are tackled. For collection of user requirements, a novel approach through an iterative design interface is proposed. In addition, fuzzy interference-based approach is proposed to capture users' biases and expertise for guiding their resource selection for their applications. The results showed improvement in performance i.e. time to execute in 98 percent of the studied applications. The data collected on user's requirements and preferences is later used by optimizer engine and machine learning algorithms for resource brokering. For resource brokering, a new integer linear programming based solution (OnTimeURB) is proposed which creates multi-cloud template solutions for resource allocation while also optimizing performance, agility, cost, and security. The solution was further improved by the addition of a machine learning model based on naive bayes classifier which captures the true QoS of cloud resources for guiding template solution creation. The proposed solution was able to improve the time to execute for as much as 96 percent of the largest applications. As discussed above, to fulfill necessity of economical computing resources, a new paradigm of computing viz-a-viz Volunteer Edge Computing (VEC) is proposed which reduces cost and improves performance and security by creating edge clusters comprising of volunteered computing resources close to users. The initial results have shown improved time of execution for application workflows against state-of-the-art solutions while utilizing only the most secure VEC resources. Consequently, we have utilized reinforcement learning based solutions to characterize volunteered resources for their availability and flexibility towards implementation of security policies. The characterization of volunteered resources facilitates efficient allocation of resources and scheduling of workflows tasks which improves performance and throughput of workflow executions. VEC architecture is further validated with state-of-the-art bioinformatics workflows and manufacturing workflows.

show abstract

OnTimeURB: Multi-Cloud Resource Brokering for Bioinformatics Workflows

Cited by 9 publications

References 12 publications

Bioinformatics for plant and agricultural discoveries in the age of multiomics: A review and case study of maize nodal root growth under water deficit

Bioinformatics for plant and agricultural discoveries in the age of multiomics: A review and case study of maize nodal root growth under water deficit

Recommender‐as‐a‐service with chatbot guided domain‐science knowledge discovery in a science gateway

Trusted resource allocation in volunteer edge-cloud computing for scientific applications

Contact Info

Product

Resources

About