FAST: Flash-aware external sorting for mobile database systems

Automatic synthesis of out-of-core algorithms

“…Their descriptions can be found in the standard text books like [23]. More recently, effort has been expended on designing algorithms for flash memory [21,18,5], the intricate memory hierarchies of graphics cards [9,13,25,27], and multi-level memory hierarchies [16]. These papers demonstrate that the state of the art in developing out-of-core algorithms is to manually carry out ad-hoc effort; one cannot yet rely on automation or a clear design methodology.…”

“…This is particularly true for data-intensive computations. The research literature describes numerous out-of-core algorithms designed and optimized for a variety of hardware and storage device configurations [23,13,9,25,5,21,27,16,18]. These are case studies of how understanding memory hierarchies and data locality can drive algorithm design.…”

Fast sorting on flash memory sensor nodes

“…Query processing on sensor nodes has been studied [9], including issues on local data processing and in-network aggregation and data processing. There have been several algorithms proposed for sorting on flash memory [2,3,10] which are designed to do more reads instead of writes due to the asymmetric costs. These algorithms are summarized below including performance formulas.…”

“…The heap sort algorithm, called FAST(1) [10], uses a binary heap of size N tuples to store the smallest N tuples during each scan. Another value, last, is maintained which stores the largest sort key output so far.…”

“…One complication is handling duplicate sort key values. The solution [10] is to remember both the last value and the integer record number of the last tuple output. During a scan, a tuple is only considered if its key is greater than last or its key is equal to last and its record number is larger.…”