Optimized Data Layout for Spatio-temporal Data in Time Domain Astronomy

Yan, Jie; Yu, Ce; Sun, Chao; Shang, Zhaohui; Hu, Yi; Feng, Jinghua; Sun, Jizhou; Xiao, Jian

doi:10.1007/978-3-319-65482-9_30

Cited by 6 publications

(1 citation statement)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Only a few studies have considered the optimization of low-cost storage, and the general solution is to aggregate the correlated data together. Some researchers have identified the correlated data using historical data access [9], whereas others used analysis of the data attributes [16]. These approaches are preferable to real-time service systems; however, they are not suitable for the long-term archive in which the data are rarely accessed.…”

Section: Data-placement Approachesmentioning

confidence: 99%

A Redistribution Tool for Long-Term Archive of Astronomical Observation Data

Sun,

Yu,

Cui

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Astronomical observation data require long-term preservation, and the rapid accumulation of observation data makes it necessary to consider the cost of long-term archive storage. In addition to low-speed disk-based online storage, optical disk or tape-based offline storage can be used to save costs. However, for astronomical research that requires historical data (particularly time-domain astronomy), the performance and energy consumption of data-accessing techniques cause problems because the requested data (which are organized according to observation time) may be located across multiple storage devices. In this study, we design and develop a tool referred to as AstroLayout to redistribute the observation data using spatial aggregation. The core algorithm uses graph partitioning to generate an optimized data placement according to the original observation data statistics and the target storage system. For the given observation data, AstroLayout can copy the long-term archive in the target storage system in accordance with this placement. An efficiency evaluation shows that AstroLayout can reduce the number of devices activated when responding to data-access requests in timedomain astronomy research. In addition to improving the performance of data-accessing techniques, AstroLayout can also reduce the storage systems power consumption. For enhanced adaptability, it supports storage systems of any media, including optical disks, tapes, and hard disks.

show abstract

Section: Data-placement Approachesmentioning

confidence: 99%