Rethinking main memory OLTP recovery

Malviya, Nirmesh; Weisberg, Ariel; Madden, Samuel; Stonebraker, Michael

doi:10.1109/icde.2014.6816685

Cited by 101 publications

(86 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Given a transaction log, there is only one possible final state of the database system. Therefore, as long as the transaction log is persisted to stable storage, the database system can recover by replaying the log, a concept introduced by other deterministic systems [22,14]. In order to avoid replaying the entire history of all transactions, checkpointing is necessary.…”

Section: Logging and Recoverymentioning

confidence: 99%

Lazy evaluation of transactions in database systems

Faleiro

Thomson

Abadi

2014

Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data

View full text Add to dashboard Cite

Existing database systems employ an eager transaction processing scheme-that is, upon receiving a transaction request, the system executes all the operations entailed in running the transaction (which typically includes reading database records, executing userspecified transaction logic, and logging updates and writes) before reporting to the client that the transaction has completed.We introduce a lazy transaction execution engine, in which a transaction may be considered durably completed after only partial execution, while the bulk of its operations (notably all reads from the database and all execution of transaction logic) may be deferred until an arbitrary future time, such as when a user attempts to read some element of the transaction's write-set-all without modifying the semantics of the transaction or sacrificing ACID guarantees. Lazy transactions are processed deterministically, so that the final state of the database is guaranteed to be equivalent to what the state would have been had all transactions been executed eagerly.Our prototype of a lazy transaction execution engine improves temporal locality when executing related transactions, reduces peak provisioning requirements by deferring more non-urgent work until off-peak load times, and reduces contention footprint of concurrent transactions. However, we find that certain queries suffer increased latency, and therefore lazy database systems may not be appropriate for read-latency sensitive applications.

show abstract

Section: Logging and Recoverymentioning

confidence: 99%

Lazy evaluation of transactions in database systems

Faleiro

Thomson

Abadi

2014

Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data

View full text Add to dashboard Cite

show abstract

“…In addition, the logs in embedded systems record the behaviors of embedded systems, and researchers use different logs to design different recovery strategies, such as partition log [14], real-time log [15], remote log [16], and operation log [17]. However, these strategies only take the requirement of real-time into consideration, and ignore other specific requirements in embedded systems, so they cannot be applied to the embedded environment efficiently.…”

Section: Reviewmentioning

confidence: 99%

Research on recovery strategy in embedded real-time main memory databases

Tan

Yin

2016

J Embedded Systems

View full text Add to dashboard Cite

In order to recover data from embedded real-time main memory databases effectively and efficiently, this paper proposes a real-time log-based recovery approach. With respect to the real-time requirement in embedded systems, we classify the consistency in real-time main memory databases into data and transaction consistencies, analyze them theoretically, design rules for correct recovery strategy, and propose real-time log-based recover algorithms for different types of transactions. The experiments show that the proposed approach is more effective and efficient than methods in both traditional and eXtremeDB database systems.Keywords: Embedded system, Real-time main memory database, Recovery strategy, Consistency ReviewWith the development of embedded systems, the application of databases in embedded systems [1] is a hotspot in both industry and academia. Embedded systems work in an environment without manual intervention, so when a fault occurs in these systems, they need to diagnose the fault and recover it automatically all by themselves [2]. The main memory databases [3,4] can reduce the I/O operations greatly while running, and satisfy the real-time requirement of embedded systems, so the databases implemented in embedded systems usually work in the main memory.In real-time main memory databases [5][6][7], the main copy of database works in the volatile RAM, and the data is very vulnerable, so the recovery is necessary. Meantime, the I/O operations in real-time main memory databases are few, and recovery is the only part that affects the I/O performance, so the performance of recovery is critical for real-time main memory databases [8,9]. While recovering from a fault, real-time main memory databases need to satisfy multiple constraints [10,11], and this pose a huge challenge for designing reasonable recovering strategies.Checkpoint or memory snapshots [12, 13] is a commonly used program recovery strategy, but the overhead of storing states of running program is very high, and it is not suitable for embedded applications. In addition, the logs in embedded systems record the behaviors of embedded systems, and researchers use different logs to design different recovery strategies, such as partition log [14], real-time log [15], remote log [16], and operation log [17]. However, these strategies only take the requirement of real-time into consideration, and ignore other specific requirements in embedded systems, so they cannot be applied to the embedded environment efficiently. In addition, the method proposed in [13], studied the recovery strategy in main-memory, but the method is based on virtual memory snapshots. In order to improve real-time ability, Levy and Silberschatz [18] proposed an incremental recovery strategy in mainmemory database.In this paper, we analyze the consistency constraints in embedded real-time main memory databases from the perspectives of both data and transaction. Then we design some rules that an efficient recovery strategy must obey in embedded real-time main memory database...

show abstract

“…In the single-sited case, H-Store runs all transactions serially, removing the need for fine-grained locks and latches. Fault tolerance is achieved through a technique that combines command logging and periodic snapshotting [7]. S-Store inherits all of these core features and extends them in non-trivial ways.…”

Section: S-store System Overviewmentioning

confidence: 99%

S-Store

Çetintemel

Kraska

et al. 2014

Proc. VLDB Endow.

Self Cite

View full text Add to dashboard Cite

First-generation streaming systems did not pay much attention to state management via ACID transactions (e.g., [3,4]). S-Store is a data management system that combines OLTP transactions with stream processing. To create S-Store, we begin with H-Store, a main-memory transaction processing engine, and add primitives to support streaming. This includes triggers and transaction workflows to implement push-based processing, windows to provide a way to bound the computation, and tables with hidden state to implement scoping for proper isolation. This demo explores the benefits of this approach by showing how a naïve implementation of our benchmarks using only H-Store can yield incorrect results. We also show that by exploiting push-based semantics and our implementation of triggers, we can achieve significant improvement in transaction throughput. We demo two modern applications: (i) leaderboard maintenance for a version of "American Idol", and (ii) a city-scale bicycle rental scenario.

show abstract

Rethinking main memory OLTP recovery

Cited by 101 publications

References 22 publications

Lazy evaluation of transactions in database systems

Lazy evaluation of transactions in database systems

Research on recovery strategy in embedded real-time main memory databases

S-Store

Contact Info

Product

Resources

About