Detection and recovery techniques for database corruption

Abstract

Increasingly, for extensibility and performance, special purpose application code is being integrated with database system code. Such application code has direct access to database system buffers, and as a result, the danger of data being corrupted due to inadvertent application writes is increased. Previously proposed hardware techniques to protect from corruption require system calls, and their performance depends on details of the hardware architecture. We investigate an alternative approach which uses codewords associated with regions of data to detect corruption and to prevent corrupted data from being used by subsequent transactions. We develop several such techniques which vary in the level of protection, space overhead, performance, and impact on concurrency. These techniques are implemented in the Dalí main-memory storage manager, and the performance impact of each on normal processing is evaluated. Novel techniques are developed to recover when a transaction has read corrupted data caused by a bad write and gone on to write other data in the database. These techniques use limited and relatively low-cost logging of transaction reads to trace the corruption and may also prove useful when resolving problems caused by incorrect data entry and other logical errors.

PROsaiq: A Smart Device-Based and EMR-Integrated System for Patient-Reported Outcome Measurement in Routine Cancer Care

“…The MOSAIQ documentation describing HL7 V2.x import into CA provides minimal specific details about Table Items import ( [33], Section 11.27), which was our preferred method for the import and population of ePRO surveys in MOSAIQ. The MOSAIQ HL7 import provides error and identity checking functions making it a clean and robust import method without the risk of data corruption associated with the direct database write access alternative [34]. Fortunately, discussions with the MOSAIQ Help desk revealed that Table Item import is possible.…”

A survey on data security in data warehousing: Issues, challenges and opportunities

“…Other systems use these features and add encryption techniques for distributing storage [25]. Architectures for damage assessment and self-healing databases have also been proposed [9,11,12,23,24]. Although strongly effective for availability purposes, data replication techniques are always an important issue in DWs, given the volume of data and storage size typically involved.…”

Automatically detecting missing cleanup for ungraceful exits

“…However, these techniques cannot detect UE problems that may or may not be handled by exception handlers. There have been some works focusing on preventing and detecting persistency bugs in storage systems, such as le systems and database systems [1,6,7,11,16,38,45,46]. For example, if a storage system crashes during the process of writing data, the system has to carefully roll back the operation to avoid data corruption.…”