Cloud computing paradigms for pleasingly parallel biomedical applications

Gunarathne, Thilina; Wu, Tak-Lon; Choi, Jong Youl; Bae, Seunghee; Qiu, Judy

doi:10.1002/cpe.1780

Cited by 52 publications

(37 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Most of the applications are embarrassingly parallel [11]. There are some limitations to the wide adoption of the cloud paradigm for scientific computing as identified by Truong et al [23] such as the lack of cost evaluation tools, cluster machine images and, as addressed in this paper, autonomic elasticity control.…”

Section: Introductionmentioning

confidence: 91%

Efficient provisioning of bursty scientific workloads on the cloud using adaptive elasticity control

Ali-Eldin

Kihl

Tordsson

et al. 2012

Proceedings of the 3rd Workshop on Scientific Cloud Computing

View full text Add to dashboard Cite

Elasticity is the ability of a cloud infrastructure to dynamically change the amount of resources allocated to a running service as load changes. We build an autonomous elasticity controller that changes the number of virtual machines allocated to a service based on both monitored load changes and predictions of future load. The cloud infrastructure is modeled as a G/G/N queue. This model is used to construct a hybrid reactive-adaptive controller that quickly reacts to sudden load changes, prevents premature release of resources, takes into account the heterogeneity of the workload, and avoids oscillations. Using simulations with Web and cluster workload traces, we show that our proposed controller lowers the number of delayed requests by a factor of 70 for the Web traces and 3 for the cluster traces when compared to a reactive controller. Our controller also decreases the average number of queued requests by a factor of 3 for both traces, and reduces oscillations by a factor of 7 for the Web traces and 3 for the cluster traces. This comes at the expense of between 20% and 30% over-provisioning, as compared to a few percent for the reactive controller.

show abstract

Section: Introductionmentioning

confidence: 91%

Efficient provisioning of bursty scientific workloads on the cloud using adaptive elasticity control

Ali-Eldin

Kihl

Tordsson

et al. 2012

Proceedings of the 3rd Workshop on Scientific Cloud Computing

View full text Add to dashboard Cite

show abstract

“…The adoption of concurrent, cheaper and less technologically advanced hardware, compared with traditional single-thread processing, lowers the time needed for the processing of large datasets. Data storage has also become more robust, improving and facilitating redundant storage, preventing data loss, whilst allowing fault-tolerance and increased overall reliability [12,17].…”

Section: Development and Expansion Of The Big Data Technologymentioning

confidence: 99%

“…Big Data is still an emerging science that is in constant evolution, but its successful use has greatly accelerated the processing of data and improved decision making. It already has proven value in Medicine [17,18], Natural Sciences [19,20], Engineering [21], Social Sciences [22], and Legislative [9,23] fields. Examples may be drawn from the visual analysis of air quality [19], the cost-effective allocation of CO 2 emissions [24], the evaluation and identification of cost-effective acid mine drainage management [25], the forecasting of risk in criminal justice decision [26], the reduction of readmission risk in hospital patients [27], and the improvement of city governments services [28].…”

Section: Introductionmentioning

confidence: 99%

Leveraging Big Data Tools and Technologies: Addressing the Challenges of the Water Quality Sector

2017

View full text Add to dashboard Cite

Abstract:The water utility sector is subject to stringent legislation, seeking to address both the evolution of practices within the chemical/pharmaceutical industry, and the safeguarding of environmental protection, and which is informed by stakeholder views. Growing public environmental awareness is balanced by fair apportionment of liability within-sector. This highly complex and dynamic context poses challenges for water utilities seeking to manage the diverse chemicals arising from disparate sources reaching Wastewater Treatment Plants, including residential, commercial, and industrial points of origin, and diffuse sources including agricultural and hard surface water run-off. Effluents contain broad ranges of organic and inorganic compounds, herbicides, pesticides, phosphorus, pharmaceuticals, and chemicals of emerging concern. These potential pollutants can be in dissolved form, or arise in association with organic matter, the associated risks posing significant environmental challenges. This paper examines how the adoption of new Big Data tools and computational technologies can offer great advantage to the water utility sector in addressing this challenge. Big Data approaches facilitate improved understanding and insight of these challenges, by industry, regulator, and public alike. We discuss how Big Data approaches can be used to improve the outputs of tools currently in use by the water industry, such as SAGIS (Source Apportionment GIS system), helping to reveal new relationships between chemicals, the environment, and human health, and in turn provide better understanding of contaminants in wastewater (origin, pathways, and persistence). We highlight how the sector can draw upon Big Data tools to add value to legacy datasets, such as the Chemicals Investigation Programme in the UK, combined with contemporary data sources, extending the lifespan of data, focusing monitoring strategies, and helping users adapt and plan more efficiently. Despite the relative maturity of the Big Data technology and adoption in many wider sectors, uptake within the water utility sector remains limited to date. By contrast with the extensive range of applications of Big Data in in other sectors, highlight is drawn to how improvements are required to achieve the full potential of this technology in the water utility industry.

show abstract

“…V. RELATED WORK An increasing number of existing scientific applications and benchmarks have been migrated and deployed to the clouds to evaluate the performance and quality of service in the cloud environments. T. Gunarathne et al [14] compared the performances of the Cap3 and the MDS & GTM interpolation scientific applications in both EC2 and Windows Azure. X. Qiu et al [15] took a similar approach and compared the performance of 3 bioinformatics applications on Windows Azure and Dryad.…”

Section: Ii) Vm Hosting Reliabilitymentioning

confidence: 99%

Fault Tolerance and Scaling in e-Science Cloud Applications: Observations from the Continuing Development of MODISAzure

Humphrey

Cheah

et al. 2010

2010 IEEE Sixth International Conference on E-Science

View full text Add to dashboard Cite

*Abstract-It can be natural to believe that many of the traditional issues of scale have been eliminated or at least greatly reduced via cloud computing. That is, if one can create a seemingly wellfunctioning cloud application that operates correctly on small or moderatesized problems, then the very nature of cloud programming abstractions means that the same application will run as well on potentially significantly larger problems. In this paper, we present our experiences taking MODISAzure, our satellite data processing system built on the Windows Azure cloud computing platform, from the proof-of-concept stage to a point of being able to run on significantly larger problem sizes (e.g., from national-scale data sizes to global-scale data sizes). To our knowledge, this is the longest-running eScience application on the nascent Windows Azure platform. We found that while many infrastructure-level issues were thankfully masked from us by the cloud infrastructure, it was valuable to design additional redundancy and fault-tolerance capabilities such as transparent idempotent task retry and logging to support debugging of user code encountering unanticipated data issues. Further, we found that using a commercial cloud means anticipating inconsistent performance and black-box behavior of virtualized compute instances, as well as leveraging changing platform capabilities over time. We believe that the experiences presented in this paper can help future eScience cloud application developers on Windows Azure and other commercial cloud providers.

show abstract

Cloud computing paradigms for pleasingly parallel biomedical applications

Cited by 52 publications

References 15 publications

Efficient provisioning of bursty scientific workloads on the cloud using adaptive elasticity control

Efficient provisioning of bursty scientific workloads on the cloud using adaptive elasticity control

Leveraging Big Data Tools and Technologies: Addressing the Challenges of the Water Quality Sector

Fault Tolerance and Scaling in e-Science Cloud Applications: Observations from the Continuing Development of MODISAzure

Contact Info

Product

Resources

About