The Big Data Processing Algorithm for Water Environment Monitoring of the Three Gorges Reservoir Area

Zhong, Yuanchang; Zhang, Liang; Xing, Shaojing; Li, Fachuan; Wan, Beili

doi:10.1155/2014/698632

Cited by 18 publications

(5 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The dimensionality curse is a common phenomenon while dealing with big data which needs to be dealt with during data processing [188]. According to Zhong et al [189], big data slows down the processing speed of data. Therefore, they explored improving on the existing Fuzzy c-means clustering algorithm while analysing water quality data in the Three Gorges Reservoir Area.…”

Section: Big Data Analytics In Environmental Monitoringmentioning

confidence: 99%

Integrating Wireless Remote Sensing and Sensors for Monitoring Pesticide Pollution in Surface and Groundwater

Mutunga,

Sinanovic,

Harrison

2024

Sensors

View full text Add to dashboard Cite

Water constitutes an indispensable resource crucial for the sustenance of humanity, as it plays an integral role in various sectors such as agriculture, industrial processes, and domestic consumption. Even though water covers 71% of the global land surface, governments have been grappling with the challenge of ensuring the provision of safe water for domestic use. A contributing factor to this situation is the persistent contamination of available water sources rendering them unfit for human consumption. A common contaminant, pesticides are not frequently tested for despite their serious effects on biodiversity. Pesticide determination in water quality assessment is a challenging task because the procedures involved in the extraction and detection are complex. This reduces their popularity in many monitoring campaigns despite their harmful effects. If the existing methods of pesticide analysis are adapted by leveraging new technologies, then information concerning their presence in water ecosystems can be exposed. Furthermore, beyond the advantages conferred by the integration of wireless sensor networks (WSNs), the Internet of Things (IoT), Machine Learning (ML), and big data analytics, a notable outcome is the attainment of a heightened degree of granularity in the information of water ecosystems. This paper discusses methods of pesticide detection in water, emphasizing the possible use of electrochemical sensors, biosensors, and paper-based sensors in wireless sensing. It also explores the application of WSNs in water, the IoT, computing models, ML, and big data analytics, and their potential for integration as technologies useful for pesticide monitoring in water.

show abstract

Section: Big Data Analytics In Environmental Monitoringmentioning

confidence: 99%

Integrating Wireless Remote Sensing and Sensors for Monitoring Pesticide Pollution in Surface and Groundwater

Mutunga,

Sinanovic,

Harrison

2024

Sensors

View full text Add to dashboard Cite

show abstract

“…Zhong et al [73] developed a fast fuzzy C-means clustering algorithm to analyze water environment monitoring data of the Three Gorges Reservoir Area. The hard cluster center can be treated as the initial value of the fuzzy cluster center to accelerate the speed of convergence and reduce the number of iterations.…”

Section: New Technologies For Marine Environment Monitoring and Prmentioning

confidence: 99%

Internet of Things in Marine Environment Monitoring: A Review

Guobao

Shi

Sun

et al. 2019

Sensors

158

View full text Add to dashboard Cite

Marine environment monitoring has attracted more and more attention due to the growing concern about climate change. During the past couple of decades, advanced information and communication technologies have been applied to the development of various marine environment monitoring systems. Among others, the Internet of Things (IoT) has been playing an important role in this area. This paper presents a review of the application of the Internet of Things in the field of marine environment monitoring. New technologies including advanced Big Data analytics and their applications in this area are briefly reviewed. It also discusses key research challenges and opportunities in this area, including the potential application of IoT and Big Data in marine environment protection.

show abstract

“…Meanwhile, there are real-life problems, in which massive parallelization of computations on Apache Hadoop or Spark, and the use of scalable environments, like the Cloud, brought significant improvements in performance of data processing and analysis. Big data challenge was observed and solved in various works devoted to intelligent transport and smart cities [11,19,42,43,74,75,84], water monitoring [12,22,90], social networks analysis [13,14,77], multimedia processing [72,82], internet of things (IoT) [9], social media monitoring [50], Life sciences [3,31,32,44,58,69] and disease data analysis [6,45,81], telecommunication [27], and finance [2], to mention just a few. Many hot issues in various sub-fields of bioinformatics were also solved with the use of Big Data ecosystems and Cloud computing, e.g., mapping nextgeneration sequence data to the human genome and other reference genomes, for use in a variety of biological analyzes including SNP discovery, genotyping and personal genomics [65], sequence analysis and assembly [17,30,34,35,47,62], multiple alignments of DNA and RNA sequences [86,91], codon analysis with local MapReduce aggregations [63], NGS data analysis [8], phylogeny …”

Section: Related Workmentioning

confidence: 99%

Spark-IDPP: high-throughput and scalable prediction of intrinsically disordered protein regions with Spark clusters on the Cloud

2018

View full text Add to dashboard Cite

Intrinsically disorder proteins (IDPs) constitute a significant part of proteins that exist and act in cells of living organisms. IDPs play key roles in central cellular processes and some of them are closely related to various human diseases, like cancer or neurodegenerative disorders. Identification of IDPs and studying their structural characteristics have become an important part of structural bioinformatics and structural genomics. However, growing amount of genomic and protein sequences in public repositories pose a pressure on existing methods for identification of IDPs. Large volumes of protein amino acid sequences need to be analyzed in terms of propensity to form disordered regions, and this task requires novel tools and scalable platforms to cope with this big biological data challenge. In this paper, we show how the identification of disordered regions of 3D protein structures can be efficiently accelerated with the use of Apache Spark cluster established and scaled on the public Cloud. For this purpose, we propose Spark-based meta-predictor (Spark-IDPP), which enables efficient prediction of disordered regions of proteins on a large-scale. Results of our performance tests show that, for large data sets, our method achieves almost linear speedup, when scaling out the computations on the 32-node Spark cluster located in the Azure cloud. This proves that through appropriate partitioning of data and by increasing the degree of parallelism, we can significantly improve efficiency of IDP predictions. Additionally, by using several basic predictors, aggregating their ranks in various consensus modes, and filtering the final outcome with a dedicated fuzzy filter, the Spark-IDPP increases the quality of predictions.

show abstract

The Big Data Processing Algorithm for Water Environment Monitoring of the Three Gorges Reservoir Area

Cited by 18 publications

References 18 publications

Integrating Wireless Remote Sensing and Sensors for Monitoring Pesticide Pollution in Surface and Groundwater

Integrating Wireless Remote Sensing and Sensors for Monitoring Pesticide Pollution in Surface and Groundwater

Internet of Things in Marine Environment Monitoring: A Review

Spark-IDPP: high-throughput and scalable prediction of intrinsically disordered protein regions with Spark clusters on the Cloud

Contact Info

Product

Resources

About