Supporting biodiversity studies with the EUBrazilOpenBio Hybrid Data Infrastructure

Amaral, Rafael; Badía, Rosa M.; Blanquer, Ignácio; Braga-Neto, Ricardo; Candela, Leonardo; Castelli, Donatella; Flann, Christina; Giovanni, Renato De; Gray, W. A.; Jones, Andrew C.; Lezzi, Daniele; Pagano, Pasquale; Perez-Canhos, Vanderlei; Quevedo, Francisco; Rafanell, Roger; Rebello, Vinod E. F.; Sousa‐Baena, Mariane S.; Torres, Erik

doi:10.1002/cpe.3238

Cited by 18 publications

(14 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…COMPSs is used in production at MareNostrum supercomputer and has been used to implement different real world applications, specially in the area of BioInformatics and Computational Genomics [10] [13] [17], Big Data analytics [29] and as building block for several scientific cyber-infrastructures [45] [22]. Other examples of applications developed with COMPSs can be found in [7].…”

Section: Compss Overviewmentioning

confidence: 99%

Transparent Orchestration of Task-based Parallel Applications in Containers Platforms

et al. 2018

Self Cite

View full text Add to dashboard Cite

This paper presents a framework to easily build and execute parallel applications in container-based distributed computing platforms in a usertransparent way. The proposed framework is a combination of the COMP Superscalar (COMPSs) programming model and runtime, which provides a straightforward way to develop task-based parallel applications from sequential codes, and containers management platforms that ease the deployment of applications in computing environments (as Docker, Mesos or Singularity). This framework provides scientists and developers with an easy way to implement parallel distributed applications and deploy them in a one-click fashion. We have built a prototype which integrates COMPSs with different containers engines in different scenarios: i) a Docker cluster, ii) a Mesos cluster, and iii) Singularity in an HPC cluster. We have evaluated the overhead in the building phase, deployment and execution of two benchmark applications compared to a Cloud testbed based on KVM and OpenStack and to the usage of bare metal nodes. We have observed an important gain in comparison to cloud environments during the building and deployment phases. This enables better adaptation of resources with respect to the computational load. In contrast, we detected an extra overhead during the execution, which is mainly due to the multi-host Docker networking.

show abstract

Section: Compss Overviewmentioning

confidence: 99%

Transparent Orchestration of Task-based Parallel Applications in Containers Platforms

et al. 2018

Self Cite

View full text Add to dashboard Cite

show abstract

“…Moreover, it offers facilities for species occurrence datasets processing [12], facilities for species distribution modeling [13], and facilities for taxonomic and nomenclature data comparison [14,15]. The facilities species occurrence datasets processing include algebraic operations (union, intersection, subtraction, and duplicates deletion) based on spatial and syntactic similarity measurements, clustering (e.g., density based algorithms such as DBScan, distance based algorithms such as K-means), outliers detection (e.g., Local Outlier Factor approach), occurrence points representativeness (e.g., Habitat Representativeness Score technique), and occurrence points enrichment with chemical and physical environmental parameters.…”

Section: Biolcubementioning

confidence: 99%

Realising Virtual Research Environments by Hybrid Data Infrastructures: the D4Science Experience

Candela¹,

Pagano²,

Castelli³

et al. 2014

Proceedings of International Symposium on Grids and Clouds (ISGC) 2014 — PoS(ISGC2014)

Self Cite

View full text Add to dashboard Cite

Modern science tend to be more than ever multidisciplinary, collaborative and demanding. Scientific investigations span the boundaries of single institutions, disciplines, and countries. In order to serve these scenarios innovative working environments are needed. In this paper it is presented an innovative environment consisting of an Hybrid Data Infrastructure supporting the dynamic deployment and operation of an array of Virtual Research Environments, each tailored to serve the needs of a scientific community towards a research endeavour. The paper reports on the enabling technology and how it has been deployed to realise the D4Science infrastructure and serve the needs of different communities.

show abstract

“…For this reason, Web services currently permeate Web development, as can be seen by the fact that most major Web sites provide some mechanism for other programs to access their data or use their functionality. This allows all kinds of user interfaces and other applications to be developed on top of standard Application Programming Interfaces (APIs) created for remote calls, making Web services important components of complex cyberinfrastructures (see Stein 2008 andAmaral et al 2014 for examples). Moreover, service-based applications can free users from the burden of configuring more specific computing resources.…”

mentioning

confidence: 99%

“…OMWS should therefore only be seen as a remote niche modelling engine service, not a repository. Model repositories, such as the EUBrazilOpenBio niche modelling application (Amaral et al 2014) . Although this clearly facilitates using openModeller tools in the background, by no means it prevents other niche modelling tools to be used.…”

mentioning

confidence: 99%

“…More recently, BioGeo, through the Brazilian Virtual Herbarium, is using a separate OMWS server to process all ENM requests. The EUBrazilOpenBio project created a Web interface for ENM where users can create complex experiments involving multiple species, algorithms and environmental scenarios to be processed by an OMWS2 service through the new runExperiment operation (Amaral et al 2014). All ENM workflows created as part of the BioVeL project 13 also interact with an OMWS2 service.…”

mentioning

confidence: 99%

See 1 more Smart Citation

OMWS: A Web Service Interface for Ecological Niche Modelling

et al. 2015

Self Cite

View full text Add to dashboard Cite

Abstract.-Ecological niche modelling (ENM) experiments often involve a high number of tasks to be performed. Such tasks may consume a significant amount of computing resources and take a long time to complete, especially when using personal computers. OMWS is a Web service interface that allows more powerful computing back-ends to be remotely exploited by other applications to carry out ENM tasks. Its latest version includes a new operation that can be used to specify complex workflows in a single request, adding the possibility of using workflow management systems on parallel computing back-end. In this paper we describe the OMWS protocol and compare its most recent version with the previous one by running the same ENM experiment using two functionally equivalent clients, each designed for one of the OMWS interface versions. Different back-end configurations were used to investigate how the performance scales for each protocol version when more processing power is made available. Results show that the new version outperforms (by a factor of two) the previous one when more computing resources are used.

show abstract

Supporting biodiversity studies with the EUBrazilOpenBio Hybrid Data Infrastructure

Cited by 18 publications

References 47 publications

Transparent Orchestration of Task-based Parallel Applications in Containers Platforms

Transparent Orchestration of Task-based Parallel Applications in Containers Platforms

Realising Virtual Research Environments by Hybrid Data Infrastructures: the D4Science Experience

OMWS: A Web Service Interface for Ecological Niche Modelling

Contact Info

Product

Resources

About