Dynamic Multivariate Outlier Detection Algorithm Using Ultraviolet Visible Spectroscopy for Monitoring Surface Water Contamination With Hydrological Fluctuation in Real-Time

Abstract

The contamination of surface water is of great harm. Ultraviolet-visible (UV-Vis) spectroscopy is an effective method to detect water contamination. However, surface water quality is influenced by hydrological fluctuation caused by rain, change of flow, etc., leading to changes of spectral characteristics over time. In the process of contamination detection, such changes cause confusion between hydrological fluctuation spectra and contaminated water spectra, thus increasing the false alarm rate. Besides, missing alarms of contaminated water is a common problem when the signal-to-noise ratio is low. In this paper, a dynamic multivariable outlier sampling rate detection (DM-SRD) algorithm is proposed. A dynamic updating strategy is introduced to increase adaptability to hydrological fluctuation. Additionally, multiple outlier variables are adopted as outlying degree indicators, which increases the accuracy of contamination detection. Two experiments were carried out using spectra collected from real surface water sites and hydrological fluctuation was constructed. To verify the effectiveness of the DM-SRD method, a comparison with the static SRD method and spectral match method was conducted. The results show that the accuracy of the DM-SRD method is 97.8%. Compared with the other two detection methods, DM-SRD significantly reduces false alarm rate and avoids missing alarms. Additionally, the results demonstrate that whether the database contained prior information on hydrological fluctuation or not, DM-SRD maintained high detection accuracy, which indicates great adaptability and robustness.