Towards Generic Fine-Grained Transaction Isolation in Polystores

Abstract

Transactional isolation is a challenge for polystores, as along with the limited capabilities of each datastore, we have to contend with their sheer diversity. However, transactional isolation is increasingly desirable as a variety of datastores are being sought after for roles that go beyond simple data lakes, where information is mostly static. Transactional guarantees are also relevant for reliability at scale. Finally, it would also close the gap to what is available in multi-model database systems. In this paper, we propose that transactional isolation in polystores can be achieved by leveraging the query engine, i.e., we implement some of the responsibilities of a transactional storage manager (TSM) in the query language itself. This has the key advantage of greatly simplifying design and implementation, as it doesn't need to be re-invented for each datastore, and should increase performance, by taking advantage of dynamic query optimization where available. We demonstrate the feasibility of the proposal with a simple proof-of-concept and experiment.

AIDA-DB: A Data Management Architecture for the Edge and Cloud Continuum

“…We thus propose another transactional layer that requires no modification to the edge, by leveraging a cloud cache and query-engine layer [24]. Briefly, write-sets of temporary transactions are kept in the cloud; After commit, the data gets transferred to a persistent table in the cache, still in the cloud, tagged with the respective commit timestamp.…”