Cloud Kotta: Enabling secure and scalable data analytics in the cloud

Babuji, Yadu; Chard, Kyle; Gerow, Aaron; Duede, Eamon

doi:10.1109/bigdata.2016.7840616

Cited by 13 publications

(6 citation statements)

References 24 publications

(22 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The MRDP design pattern could also be implemented using commercial cloud services, such as those provided by Amazon, Google, and Microsoft. Indeed, many public scientific datasets are now 17/22 hosted on cloud platforms and researchers are increasingly leveraging cloud services for managing and analyzing data (Babuji et al, 2016). Cloud services provide scalable data management, rich identity management, and secure authentication and authorization, all accessible via APIs-a complete set of capabilities needed to implement the MRDP design pattern.…”

Section: Related Workmentioning

confidence: 99%

The Modern Research Data Portal: A design pattern for networked, data-intensive science

Chard

Dart

Foster

et al. 2017

Preprint

Self Cite

View full text Add to dashboard Cite

We describe best practices for providing convenient, high-speed, secure access to large data via research data portals. We capture these best practices in a new design pattern, the Modern Research Data Portal, that disaggregates the traditional monolithic web-based data portal to achieve orders-of-magnitude increases in data transfer performance, support new deployment architectures that decouple control logic from data storage, and reduce development and operations costs. We introduce the design pattern; explain how it leverages high-performance Science DMZs and cloud-based data management services; review representative examples at research laboratories and universities, including both experimental facilities and supercomputer sites; describe how to leverage Python APIs for authentication, authorization, data transfer, and data sharing; and use coding examples to demonstrate how these APIs can be used to implement a range of research data portal capabilities. Sample code at a companion web site, https://docs.globus.org/mrdp, provides application skeletons that readers can adapt to realize their own research data portals.

show abstract

Section: Related Workmentioning

confidence: 99%

The Modern Research Data Portal: A design pattern for networked, data-intensive science

Chard

Dart

Foster

et al. 2017

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…A simple illustration of this model is shown in Figure 3. We then execute these steps by submitting a series of jobs to the Cloud Kotta [3] platform.…”

Section: Contextualizationmentioning

confidence: 99%

Skluma

Beckman

Skluzacek

Chard

et al. 2017

Proceedings of the 29th International Conference on Scientific and Statistical Database Management

Self Cite

View full text Add to dashboard Cite

Scientists' capacity to make use of existing data is predicated on their ability to find and understand those data. While significant progress has been made with respect to data publication, and indeed one can point to a number of well organized and highly utilized data repositories, there remain many such repositories in which archived data are poorly described and thus impossible to use. We present Skluma-an automated system designed to process vast amounts of data and extract deeply embedded metadata, latent topics, relationships between data, and contextual metadata derived from related documents. We show that Skluma can be used to organize and index a large climate data collection that totals more than 500GB of data in over a half-million files.

show abstract

“…To address the need for the secure and scalable management and analysis of data, we developed the Cloud Ko a secure data enclave [6,7] Cloud Ko a provides a cloud-based, elastically scalable environment that is able to meet the needs of sporadic and bursty scienti c analysis workloads while removing the need for owning and operating large scale infrastructure. It implements a ne grain access control model over managed research data allowing controlled access from within and outside the enclave.…”

Section: Introductionmentioning

confidence: 99%

Enabling Interactive Analytics of Secure Data using Cloud Kotta

Babuji¹,

Chard²,

Duede³

2017

Preprint

Self Cite

View full text Add to dashboard Cite

Research, especially in the social sciences and humanities, is increasingly reliant on the application of data science methods to analyze large amounts of (o en private) data. Secure data enclaves provide a solution for managing and analyzing private data. However, such enclaves do not readily support discovery science-a form of exploratory or interactive analysis by which researchers execute a range of (sometimes large) analyses in an iterative and collaborative manner. e batch computing model o ered by many data enclaves is well suited to executing large compute tasks; however it is far from ideal for day-to-day discovery science. As researchers must submit jobs to queues and wait for results, the high latencies inherent in queue-based, batch computing systems hinder interactive analysis. In this paper we describe how we have augmented the Cloud Ko a secure data enclave to support collaborative and interactive analysis of sensitive data. Our model uses Jupyter notebooks as a exible analysis environment and Python language constructs to support the execution of arbitrary functions on private data within this secure framework.

show abstract

Cloud Kotta: Enabling secure and scalable data analytics in the cloud

Cited by 13 publications

References 24 publications

The Modern Research Data Portal: A design pattern for networked, data-intensive science

The Modern Research Data Portal: A design pattern for networked, data-intensive science

Skluma

Enabling Interactive Analytics of Secure Data using Cloud Kotta

Contact Info

Product

Resources

About