Fast sorting on flash memory sensor nodes

Abstract

Sensor nodes are being used in numerous domains for data collection and analysis. The ability to perform on device data processing increases the functionality and lifetime of a network as it avoids network transmission. Previous work has developed algorithms for sorting on sensor nodes with flash memory. These algorithms favour reads over writes due to the asymmetric costs. However, previous algorithms have not exploited the ability to perform random reads at the same cost as sequential reads. In this paper, we propose a new algorithm called Flash MinSort that uses random reads to rapidly sort in flash memory using a small amount of memory. The algorithm works especially well for sensor data which is often temporally clustered. Experimental results on random and real sensor data show that Flash MinSort is two to ten times faster than previous approaches for small memory sizes where external merge sort is not executable.

Data sorting in flash memory

“…Therefore, this observation is the motivation for this paper. Previous research [4,5,6] has been proposed for flash-based storage systems. They can decrease a lot of write requests by generating longer runs but could cause additional read requests.…”

Efficient External Sorting on Flash Memory Embedded Devices

“…An initial version of Flash MinSort was presented in [1]. This paper contains a generalized version of Flash MinSort that adapts to the amount of memory available and the size of the dataset.…”