In November 2018, the KISTI Tier-1 centre started a project to design, develop and deploy a disk-based custodial storage with error rate and reliability compatible with a tape-based storage. This project has been conducted in collaboration with KISTI and CERN; especially the initial design was laid out from the intensive discussion with CERN IT and ALICE. The system design of the disk-based custodial storage consisted of high density JBOD enclosures and erasure coding data protection, implemented in EOS, the open-source storage management developed at CERN. In order to balance the system reliability, data security and I/O performance, we investigated the possible SAS connections of JBOD enclosures to the front-end node managed by EOS and the technology constraints of interconnections in terms of throughput to accommodate large number of disks foreseen in the storage. This project will be completed and enter production before the start of LHC Run3 in 2021. In this paper we present the detailed description on the initial system design, the brief results of test equipment for the procurement, the deployment of the system, and the further plans for the project.
We proposed a disk-based custodial storage as an alternative to tape for the ALICE experiment at CERN to preserve its raw data. The proposed storage system relies on Redundant Array of Independent Nodes (RAIN) layout – the implementation of erasure coding in the EOS storage suite, which is developed by CERN – for data protection and takes full advantage of high-density Just-Bunch-Of-Disks (JBOD) enclosures to maximize storage capacity as well as to achieve cost-effectiveness comparable to tape. The system we present provides 18 PB of total raw capacity from the 18 set of high-density JBOD enclosures attached to 9 EOS front-end servers. In order to balance between usable space and data protection, the system will stripe a file into 16 chunks on the 4-parity enabled RAIN layout configured on top of 18 containerized EOS FSTs. Although the reduction rate of available space increases up to 33:3% with this layout, the estimated annual data loss rate drops down to 8:6 × 10−5%. In this paper, we discuss the system architecture of the disk-based custodial storage, 4-parity RAIN layout, deployment automation, and the integration to the ALICE experiment in detail.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
customersupport@researchsolutions.com
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.