NEWT: A RESTful service for building High Performance Computing web applications

Cholia, Shreyas; Skinner, David; Boverhof, Joshua

doi:10.1109/gce.2010.5676125

Cited by 46 publications

(30 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…They provide access to a wide range of methods and tools using SOAP as well as RESTful web services. Cholia et al [7] propose NEWT, a RESTful web service for high performance applications that provides access to methods hosted at the National Energy Research Scientific Computing Center. They provide access to command line tools for researchers who are more comfortable using web services.…”

Section: Related Workmentioning

confidence: 99%

SDK Reinvented: Document Image Analysis Methods as RESTful Web Services

Würsch

Ingold

Liwicki

2016

2016 12th IAPR Workshop on Document Analysis Systems (DAS)

View full text Add to dashboard Cite

Document Image Analysis (DIA) systems become ever more advanced, but also more complex -computationally, and logically. This increases the difficulty of integrating existing state-of-the-art approaches into new research or into practical workflows. The current approach to sharing software is publishing source code -leaving the burden to the integratoror creating a Software Development Kit (SDK) which is often restricted to one programming language.We present DIVAServices a framework for sharing and accessing DIA methods within the research community and beyond. Using a RESTful web service architecture we provide access to the methods, leading to only one system on which the binaries of methods need to be maintained. All it takes for a developer to use an algorithm is a simple HTTP request with the image data and parameters for the method and they will receive the computed results in a format that allows for seamless integration into any kind of workflow or for further processing. Furthermore, DIVAServices is open-source, enabling other research groups or libraries to host their own instance in their environment. Using this framework, future DIA systems can be built on the shoulders of well tested algorithms, accessible to everyone.

show abstract

Section: Related Workmentioning

confidence: 99%

SDK Reinvented: Document Image Analysis Methods as RESTful Web Services

Würsch

Ingold

Liwicki

2016

2016 12th IAPR Workshop on Document Analysis Systems (DAS)

View full text Add to dashboard Cite

show abstract

“…The NEWT [3] project also provides a RESTful-based Web service interface to High-Performance Computing (HPC) systems. NEWT is focused on providing access to a particular set of resources (NERSC), and does not address the need for a distributed set of (virtualized) computing resources to be interconnected by overlay virtual networks.…”

Section: Related Workmentioning

confidence: 99%

GRAPLEr: A distributed collaborative environment for lake ecosystem modeling that integrates overlay networks, high‐throughput computing, and WEB services

Subratie

Aditya

Mahesula

et al. 2017

Concurrency and Computation

View full text Add to dashboard Cite

The GLEON Research And PRAGMA Lake Expedition -GRAPLE -is a collaborative effort between computer science and lake ecology researchers. It aims to improve our understanding and predictive capacity of the threats to the water quality of our freshwater resources, including climate change. This paper presents GRAPLEr, a distributed computing system used to address the modeling needs of GRAPLE researchers. GRAPLEr integrates and applies overlay virtual network, high-throughput computing, and Web service technologies in a novel way. First, its user-level IP-over-P2P (IPOP) overlay network allows compute and storage resources distributed across independently-administered institutions (including private and public clouds) to be aggregated into a common virtual network, despite the presence of firewalls and network address translators. Second, resources aggregated by the IPOP virtual network run unmodified high-throughput computing middleware (HTCondor) to enable large numbers of model simulations to be executed concurrently across the distributed computing resources. Third, a Web service interface allows end users to submit job requests to the system using client libraries that integrate with the R statistical computing environment. The paper presents the GRAPLEr architecture, describes its implementation and reports on its performance for batches of General Lake Model (GLM) simulations across three cloud infrastructures (University of Florida, CloudLab, and Microsoft Azure).

show abstract

“…These VMs are run using qemu and Linux Kernel-based Virtual Machine (https://www.linux-kvm.org) on top of the MIT SuperCloud system image and can share a compute node side by side with other non-VM jobs. These VMs can utilize Virtual Distributed Ethernet [Davoli 2005] and VXLAN [RFC 7348] to create private networks between VMs and VirtFS folder sharing based on Plan 9 [Jujjuri 2010] to mount the HPC storage through the hypervisor as the user that launched the VM. With these building blocks, a user can simulate an entire enterprise of interconnected desktops, servers and networks, or run a computational program that requires a legacy operating system with minimal sacrifice of access to the large HPC storage.…”

Section: B Mit Supercloud Virtal Machine Systemmentioning

confidence: 99%

MIT SuperCloud portal workspace: Enabling HPC web application deployment

Prout

Arcand

Bestor

et al. 2017

2017 IEEE High Performance Extreme Computing Conference (HPEC)

View full text Add to dashboard Cite

The MIT SuperCloud Portal Workspace enables the secure exposure of web services running on high performance computing (HPC) systems. The portal allows users to run any web application as an HPC job and access it from their workstation while providing authentication, encryption, and access control at the system level to prevent unintended access. This capability permits users to seamlessly utilize existing and emerging tools that present their user interface as a website on an HPC system creating a portal workspace.Performance measurements indicate that the MIT SuperCloud Portal Workspace incurs marginal overhead when compared to a direct connection of the same service.

show abstract

NEWT: A RESTful service for building High Performance Computing web applications

Cited by 46 publications

References 7 publications

SDK Reinvented: Document Image Analysis Methods as RESTful Web Services

SDK Reinvented: Document Image Analysis Methods as RESTful Web Services

GRAPLEr: A distributed collaborative environment for lake ecosystem modeling that integrates overlay networks, high‐throughput computing, and WEB services

MIT SuperCloud portal workspace: Enabling HPC web application deployment

Contact Info

Product

Resources

About