The GLEON Research And PRAGMA Lake Expedition -GRAPLE -is a collaborative effort between computer science and lake ecology researchers. It aims to improve our understanding and predictive capacity of the threats to the water quality of our freshwater resources, including climate change. This paper presents GRAPLEr, a distributed computing system used to address the modeling needs of GRAPLE researchers. GRAPLEr integrates and applies overlay virtual network, high-throughput computing, and Web service technologies in a novel way. First, its user-level IP-over-P2P (IPOP) overlay network allows compute and storage resources distributed across independently-administered institutions (including private and public clouds) to be aggregated into a common virtual network, despite the presence of firewalls and network address translators. Second, resources aggregated by the IPOP virtual network run unmodified high-throughput computing middleware (HTCondor) to enable large numbers of model simulations to be executed concurrently across the distributed computing resources. Third, a Web service interface allows end users to submit job requests to the system using client libraries that integrate with the R statistical computing environment. The paper presents the GRAPLEr architecture, describes its implementation and reports on its performance for batches of General Lake Model (GLM) simulations across three cloud infrastructures (University of Florida, CloudLab, and Microsoft Azure).
The success and scale of the Internet and its protocol IP has spurred emergent distributed technologies such as fog/edge computing and new application models based on distributed containerized microservices. The Internet of Things and Connected Communities are poised to build on these technologies and models and to benefit from the ability to communicate in a peer-to-peer (P2P) fashion. Ubiquitous sensing, actuating and computing implies a scale that breaks the centralized cloud computing model. Challenges stemming from limited IPv4 public addresses, the need for transport layer authentication, confidentiality and integrity become a burden on developing new middleware and applications designed for the network's edge. One approach -not reliant on the slow adoption of IPv6 -is the use of virtualized overlay networks, which abstract the complexities of the underlying heterogeneous networks that span the components of distributed fog applications and middleware. This paper describes the evolution of the design and implementation of IP-over-P2P (IPOP) -from its purist P2P inception, to a pragmatic hybrid model which is influenced by and incorporates standards. The hybrid client-server/P2P approach allows IPOP to leverage existing robust and mature cloud infrastructure, while still providing the characteristics needed at the edge. IPOP is networking cyber infrastructure that presents an overlay virtual private network which selforganizes with dynamic membership of peer nodes into a scalable structure. IPOP is resilient to partitioning, supports redundant paths within its fabric, and provides software defined programming of switching rules to utilize these properties of its topology.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.