HDFS Efficiency Storage Strategy for Big Data in Smart City

Xiang, Min; Jiang, Yuzhou; Xia, Zhong; Xu, Longzhang; Huang, Chao

doi:10.1109/cac48633.2019.8996571

Cited by 3 publications

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Rathidevi and Parameswari [22] proposed a centroid-based clustering of the small files method, which placed related files in clusters to improve data access performance. Xiang et al [23] proposed a Hadoop distributed file system storage strategy based on the response time of data nodes, and the simulation results show that this strategy can realize the distributed storage of big data and avoid the emergence of hot nodes. In summary, the current research on medical imaging big data storage ignores the characteristics of clinical diagnosis as a unit of examination.…”

Section: Medical Big Data Storage and Accessmentioning

confidence: 99%

Hadoop-Based Medical Image Storage and Access Method for Examination Series

Huang

Wang³

et al. 2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Under the background of electronic medical data, doctors use electronic images to replace the traditional film for diagnosis, and patients can view examination images at any time through various electronic means. The storage and frequent reading of massive data bring new challenges. Given the characteristics of the size and quantity of image files generated by different examination types, different merging strategies are proposed to improve the storage performance of the files; according to the characteristics of medical data with examination as the basic unit, a two-level model combined with medical imaging information is proposed. The indexing mechanism solves the problem that SEQ files cannot be read randomly without an index; given the time characteristics of data access, an improved 2Q algorithm is proposed to cache the prefetched files and the read files in different cache queues, which improves the efficiency of file reading. In the experimental comparison, the proposed algorithm surpasses the baseline method in storage and access performance.

show abstract

Section: Medical Big Data Storage and Accessmentioning

confidence: 99%

Hadoop-Based Medical Image Storage and Access Method for Examination Series

Huang

Wang³

et al. 2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

show abstract

A distributed cache system based on Redis for high-speed railway catenary monitoring system

Chen

Wang²,

et al. 2020

2020 Chinese Automation Congress (CAC)

View full text Add to dashboard Cite

A Cloud Computing-Based Approach for Efficient Processing of Massive Machine Tool Diagnosis Data

Zhang

Tao

2021

J CIRCUIT SYST COMP

View full text Add to dashboard Cite

This paper proposes a cloud computing-based approach to efficiently process the massive data produced in intelligent machine tool diagnosis flow. By collecting and extracting the vibration, power and other useful system signals during the machining operation of machine tools, the cutting process samples and cutting gap samples of machine tools can be accurately segmented, in order to construct a set of signal samples that can effectively and completely characterize the level of tool wear. We propose a visual detection method that relies on local threshold segmentation to predict tool wear status. The machine tool image is divided into several small blocks, and each image block is segmented to obtain the segmentation threshold, which is defined as the local threshold of each block. Then, the detection method scans the whole image based on the maximum local threshold among all blocks. Considering the complicated flow of visual detection and the high volume of machine tool diagnosis data, we further propose a big data processing approach which is implemented on a cloud computing architecture. By modeling the workflow of the proposed visual detection method as a directed acyclic graph, we develop a scheduling model that aims at minimizing the execution time of massive tool diagnosis data processing with available cloud computing resources. A effective metaheuristic based on search strategy of artificial bee colony is developed to solve the formulation scheduling problem. Experimental results on a cloud-based system demonstrate that, the visual detection method enhances the accuracy of tool wear detection, and the cloud-based approach significantly reduces the execution time of tool diagnosis flow by means of distributed computing.

show abstract

HDFS Efficiency Storage Strategy for Big Data in Smart City

Cited by 3 publications

References 0 publications

Hadoop-Based Medical Image Storage and Access Method for Examination Series

Hadoop-Based Medical Image Storage and Access Method for Examination Series

A distributed cache system based on Redis for high-speed railway catenary monitoring system

A Cloud Computing-Based Approach for Efficient Processing of Massive Machine Tool Diagnosis Data

Contact Info

Product

Resources

About