Serverless OpenHealth at data commons scale—traversing the 20 million patient records of New York’s SPARCS dataset in real-time

Almeida, Jonas S.; Hajagos, Janos; Saltz, Joel; Saltz, Mary

doi:10.7717/peerj.6230

Cited by 6 publications

(2 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[ 20 ] It is also informed by our recent work on granular orchestration of stateless Application Programming Interfaces at Data Commons scale. [ 21 ] The test results below illustrate a use case where, in addition to the advantageous security and scalability, the costs associated with the proposed solution are up to one hundredth of the conventional approach [ Table 1 ].…”

Section: Onclusionsmentioning

confidence: 99%

ImageBox 2 – Efficient and Rapid Access of Image Tiles from Whole-Slide Images Using Serverless HTTP Range Requests

Bremer

Saltz

Almeida

2020

Journal of Pathology Informatics

Self Cite

View full text Add to dashboard Cite

Background: Whole-slide images (WSI) are produced by a high-resolution scanning of pathology glass slides. There are a large number of whole-slide imaging scanners, and the resulting images are frequently larger than 100,000 × 100,000 pixels which typically image 100,000 to one million cells, ranging from several hundred megabytes to many gigabytes in size. Aims and Objectives: Provide HTTP access over the web to Whole Slide Image tiles that do not have localized tiling servers but only basic HTTP access. Move all image decode and tiling functions to calling agent (ImageBox). Methods: Current software systems require tiling image servers to be installed on systems providing local disk access to these images. ImageBox2 breaks this requirement by accessing tiles from remote HTTP source via byte-level HTTP range requests. This method does not require changing the client software as the operation is relegated to the ImageBox2 server which is local (or remote) to the client and can access tiles from remote images that have no server of their own such as Amazon S3 hosted images. That is, it provides a data service [on a server that does not need to be managed], the definition of serverless execution model increasingly favored by cloud computing infrastructure. Conclusions: The specific methodology described and assessed in this report preserves normal client connection semantics by enabling cloud-friendly tiling, promoting a web of http connected whole-slide images from a wide-ranging number of sources, and providing tiling where local tiling servers would have been otherwise unavailable.

show abstract

Section: Onclusionsmentioning

confidence: 99%

ImageBox 2 – Efficient and Rapid Access of Image Tiles from Whole-Slide Images Using Serverless HTTP Range Requests

Bremer

Saltz

Almeida

2020

Journal of Pathology Informatics

Self Cite

View full text Add to dashboard Cite

show abstract

“…In addition, many applications using serverless computing have been proposed in the literature. For example, Almeida et al presented a novel tool [28] that used a serverless API to traverse an enormous number of medical records in a remote database. Lee et al presented a tool [25] to visualize DNA sequences, which allows users to upload a DNA sequence in FASTA format to AWS S3 where the sequence is subsequently processed by AWS Lambda.…”

Section: Related Workmentioning

confidence: 99%

Leveraging Serverless Computing to Improve Performance for Sequence Comparison

Niu

Kumanov

Hung

et al. 2019

Proceedings of the 10th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics

View full text Add to dashboard Cite

Cloud computing offers on-demand, scalable computing and storage, and has become an essential resource for the analyses of big biomedical data. The usual approach to cloud computing requires users to reserve and provision virtual servers. An emerging alternative is to have the provider allocate machine resources dynamically. This type of serverless computing has tremendous potential for biomedical research in terms of ease-of-use, instantaneous scalability, and cost effectiveness. In our proof of concept example, we demonstrate how serverless computing provides low cost access to hundreds of CPUs, on demand, with little or no setup. In particular, we illustrate that the all-against-all pairwise comparison among all unique human proteins can be accomplished in approximately 2 minutes, at a cost of less than $1, using Amazon Web Services Lambda. We also demonstrate the feasibility of our approach using Google Functions and show that the same task of pairwise protein sequence comparison can be accomplished in approximately 11.5 minutes. In contrast, running the same task on a typical laptop computer required 8.7 hours.

show abstract

MedicaidJS: a FAIR approach to real-time drug analytics

Agarwal,

Kim,

Almeida

et al. 2023

Bioinformatics Advances

View full text Add to dashboard Cite

Motivation As prescription drug prices have drastically risen over the past decade, so has the need for real-time drug tracking resources. In spite of increased public availability to raw data sources, individual drug metrics remain concealed behind intricate nomenclature and complex data models. Some web applications, such as GoodRX, provide insight into real-time drug prices but offer limited interoperability. To overcome both obstacles we pursued the direct programmatic operation of the stateless Application Programming interfaces (HTTP REST APIs) maintained by the Food and Drug Administration (FDA), Medicaid, and National Library of Medicine. These data-intensive resources represent an opportunity to develop Software Development Kits (SDK) to streamline drug metrics without downloads or installations, in a manner that addresses the FAIR principles for stewardship in scientific data—Findability, Accessibility, Interoperability, and Reusability. These principles provide a guideline for continual stewardship of scientific data. Results MedicaidJS SDK was developed to orchestrate API calls to three complementary data resources: Medicaid (data.medicaid.gov), Food and Drug Administration (open.fda.gov), and the National Library of Medicine RxNorm (lhncbc.nlm.nih.gov/RxNav). MedicaidJS synthesizes response data from each platform into a zero-footprint JavaScript modular library that provides data wrangling, analysis, and generation of embeddable interactive visualizations. The SDK is served on github with live examples on observableHQ notebooks. It is freely available and can be embedded into web applications as modules returning structured JSON data with standardized identifiers. Availability and implementation Open source code publicly available at https://github.com/episphere/medicaid, live at episphere.github.io/medicaid, supplementary interactive Observable Notebooks at observablehq.com/@medicaidsdk/medicaidsdk.

show abstract

Serverless OpenHealth at data commons scale—traversing the 20 million patient records of New York’s SPARCS dataset in real-time

Cited by 6 publications

References 15 publications

ImageBox 2 – Efficient and Rapid Access of Image Tiles from Whole-Slide Images Using Serverless HTTP Range Requests

ImageBox 2 – Efficient and Rapid Access of Image Tiles from Whole-Slide Images Using Serverless HTTP Range Requests

Leveraging Serverless Computing to Improve Performance for Sequence Comparison

MedicaidJS: a FAIR approach to real-time drug analytics

Contact Info

Product

Resources

About