Computational storage: an efficient and scalable platform for big data and HPC applications

Abstract

In the era of big data applications, the demand for more sophisticated data centers and high-performance data processing mechanisms is increasing drastically. Data are originally stored in storage systems. To process data, application servers need to fetch them from storage devices, which imposes the cost of moving data to the system. This cost has a direct relation with the distance of processing engines from the data. This is the key motivation for the emergence of distributed processing platforms such as Hadoop, which move process closer to data. Computational storage devices (CSDs) push the "move process to data" paradigm to its ultimate boundaries by deploying embedded processing engines inside storage devices to process data. In this paper, we introduce Catalina, an efficient and flexible computational storage platform, that provides a seamless environment to process data in-place. Catalina is the first CSD equipped with a dedicated application processor running a full-fledged operating system that provides filesystem-level data access for the applications. Thus, a vast spectrum of applications can be ported for running on Catalina CSDs. Due to these unique features, to the best of our knowledge, Catalina CSD is the only in-storage processing platform that can be seamlessly deployed in clusters to run distributed applications such as Hadoop MapReduce and HPC applications in-place without any modifications on the underlying distributed processing framework. For the proof of concept, we build a fully functional Catalina prototype and a CSD-equipped platform using 16 Catalina CSDs to run Intel HiBench Hadoop and HPC benchmarks to investigate the benefits of deploying Catalina CSDs in the distributed processing environments. The experimental results show up to 2.2× improvement in performance and 4.3× reduction in energy consumption, respectively, for running Hadoop MapReduce benchmarks. Additionally, thanks to the Neon SIMD engines, the performance and energy efficiency of DFT algorithms are improved up to 5.4× and 8.9×, respectively.

Efficient parallel derivation of short distinguishing sequences for nondeterministic finite state machines using MapReduce

“…It presents several advantages, such as scalability, flexibility, cost-effectiveness, organized architecture, and resilience to failure. Recently, several Hadoop-based platforms have been proposed as efficient and flexible solutions for computational storage with the strategy of processing data close to where they reside [53][54][55]. Among them, we cite the lineage-aware data management (LDM) that exploits the data locality to decrease the network footprint [56,57].…”

In-storage Processing of I/O Intensive Applications on Computational Storage Drives

“…One unique feature with our CSD is that the CBDD supports file-system access by the ISP applications and the host. By the embedded Linux to mount partitions on the storage and present a file API, this makes it not only easier to port applications but also more efficient to access the data [18], [19]. The host CPU and ISP units can also communicate via the same partition, which is mounted by both the host and the ISP.…”

HyperTune

“…While the first approach is powerand cost-efficient, it provides limited processing power, and also it can negatively impact the performance of the SSD controller. In the previous works focusing on the second approach, researchers have investigated different types of dedicated processing engines such as FPGAs [14] and embedded processors [23,24]. Although FPGAs are power efficient, they have several limitations for being used in ISPs, including the challenging step of implementing an RTL design of the tasks [18], the time-consuming step of reconfiguration of FPGAs for running different tasks, and the lack of supporting file systems to allow tasks to deal with the concept of file when accessing the storage.…”