No abstract

There is a huge amount of duplicated or redundant data in current storage systems. So Data Deduplication, which uses lossless data compression schemes to minimize the duplicated data at the interfile level, has been receiving broad attention in recent years. But there are still research challenges in current approaches and storage systems, such as: how to chunking the files more efficiently and better leverage potential similarity and identity among dedicated applications; how to store the chunks effectively and reliably into secondary storage devices. In this paper, we propose ADMAD: an Application-Driven Metadata Aware De-duplication Archival Storage System, which makes use of certain meta-data information of different levels in the I/O path to direct the file partitioning into more Meaningful data Chunks (MC) to maximally reduce the inter-file level duplications. However, the chunks may be with different lengths and variable sizes, storing them into storage devices may result in a lot of fragments and involve a high percentage of random disk accesses, which is very inefficient. Therefore, in ADMAD, chunks are further packaged into fixed sized Objects as the storage units to speed up the I/O performance as well as to ease the data management.Preliminary experiments have demonstrated that the proposed system can further reduce the required storage space when compared with current methods (from 20% to near 50% according to several datasets), and largely improves the writing performance (about 50%-70% in average).

Bridge crack detection is essential to ensure bridge safety. The introduction of deep learning technology has made it possible to detect bridge cracks automatically and accurately. In this study, the Inception-Resnet-v2 algorithm was systematically improved and applied to the real-time detection of bridge cracks. We propose an end-to-end bridge crack detection model based on a convolutional neural network. This model combines the advantages of Inception convolution and residual networks, broadening the network width and alleviating the training problem of the deep network. The calculation speed is improved while still ensuring accuracy. Multi-scale feature fusion enables the network to extract contextual information of different scales, which improves the accuracy of crack recognition. The GKA (K-means clustering method based on a genetic algorithm) realizes the accurate segmentation of the target area, greatly enhances the clustering effect, and effectively improves the detection speed. In this model, large fracture datasets are used for training and testing without pre-training. The experimental results show that the performance of this method was improved in all aspects: accuracy, 99.24%; recall, 99.03%; F-measure, 98.79%; and FPS(Frames Per Second), 196. INDEX TERMS bridge crack detection, Inception-Resnet-v2, multiscale feature fusion, GKAThis work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.

No abstract

Data deduplication has recently become commonplace in most secondary storage and even in some primary storage for the capacity optimization purpose. Aside from its write performance, read performance of the deduplication storage has been gaining in significance with a wide range of its deployments. In this paper, we emphasize the importance of read performance in reconstituting a data stream from its unique and shared chunks physically dispersed over deduplication storage. We newly introduce a read performance indicator called Chunk Fragmentation Level (CFL). We also validate that the CFL is very effective to indicate read performance of deduplication storage through a developed theoretical performance model and extensive experiments. Finally, we articulate further research issues.While the data growth in our digital universe becomes explosive, it has been observed that the 70% of the worldwide digital data are duplicated. Certainly, data duplication [1] (briefly deduplication) would be extremely usefuly to mitigate demanded storage capacity by eliminating any duplicate data from a large set of data streams. A typical chunkingbased deduplication process divides a given data stream into smaller chunks of variable/fixed lengths and computes a hash value of each chunk with a cryptographic hash function, such as SHA-1 or MD5. Generally, dynamic chunking (variablelength chunks) outperforms static chunking (fixed-length chunks). A hash index table should effectively maintain a large number of hash values with their storage locations. Only if the hash value of a chunk is not found in the hash index table, the deduplicated (unique or unshared) chunk is written into the deduplication storage. Otherwise, the shared chunk will be eliminated from further physical write.Recent deduplication design has mainly focused on its write performance by: (1) improving the duplication detection and chunk existence querying efficiency with efficient chunking, faster hash indexing [2], locality-preserving index caching [1], and efficient bloom filters [3]; (2) compressing the unique chunks and performing (fixed-size) large writes through containers [4], [1] or similar structures [5].However, read performance of deduplication storage does matter. Traditionally, the read performance (mainly throughput) of deduplication storage, such as backup systems, has not gained in its significance because people thinks that the occurrence of reads is much smaller than that of writes. Having said that, the read performance becomes extremely critical in the event of entire system restoration from crash [1]. Higher read performance can shorten a recovery window time, resulting in higher system availability. In addition, since the deduplication storage has a limited capacity, it is periodically necessary to stage the backed up data streams for migration to archive storage, such as virtual tapes. This requires reconstruction of their original data streams, because the archive storage operations are typically stream-based. In fact, this staging frequency is ...