In order to solve the problems of low data storage efficiency and poor retrieval performance in forest ecological station, a method of a forest ecological station data management platform based on Internet of Things and big data sensor is proposed. The framework method designs the prepartition algorithm to ensure the uniform distribution of data in the cluster. According to the characteristics of ecological data, the RowKey is scientifically designed to realize the rapid retrieval of ecological data. The Elasticsearch index fragment placement strategy based on index data and server performance evaluation is designed, and the packaging and merging strategy based on data site and time correlation is proposed to improve the storage efficiency. The results are as follows: when the scale of structured data is 108, the retrieval time of the system is 1.045 s, which is 3.99 times faster than that of the original HBase. When the scale of unstructured data is 107, the packaging small image strategy based on data site and time correlation is 1.15 times higher than that of the sequence file and 1.79 times higher than that of the original HBase. In the case of 104 concurrent users, the number of queries per second after optimization is 1.88 times higher than the original, the throughput per second is 1.74 times higher than that before optimization, and the system response time is 69.5% lower than that before optimization. The results show that the proposed scheme has significantly improved the performance in the aspects of cluster load balancing and massive structured and unstructured data retrieval efficiency and system throughput, and provide the necessary theoretical basis and technical implementation for the storage and management of forest ecological data.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.