A Bayesian Framework for the Automated Online Assessment of Sensor Data Quality

Smith, Daniel; Timms, Greg; Souza, P. A. de; D’Este, Claire

doi:10.3390/s120709476

Cited by 23 publications

(25 citation statements)

References 34 publications

(57 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Once our sensor raises a malfunction alert, intelligent sensor malfunction detection techniques [2,18] can be activated to further classify the malfunction alert or discard it as a false positive.…”

Section: Discussionmentioning

confidence: 99%

Active Self-Testing Noise Measurement Sensors for Large-Scale Environmental Sensor Networks

Domínguez

Reinoso²,

Touhafi

et al. 2013

Sensors

View full text Add to dashboard Cite

Large-scale noise pollution sensor networks consist of hundreds of spatially distributed microphones that measure environmental noise. These networks provide historical and real-time environmental data to citizens and decision makers and are therefore a key technology to steer environmental policy. However, the high cost of certified environmental microphone sensors render large-scale environmental networks prohibitively expensive. Several environmental network projects have started using off-the-shelf low-cost microphone sensors to reduce their costs, but these sensors have higher failure rates and produce lower quality data. To offset this disadvantage, we developed a low-cost noise sensor that actively checks its condition and indirectly the integrity of the data it produces. The main design concept is to embed a 13 mm speaker in the noise sensor casing and, by regularly scheduling a frequency sweep, estimate the evolution of the microphone's frequency response over time. This paper presents our noise sensor's hardware and software design together with the results of a test deployment in a large-scale environmental network in Belgium. Our middle-range-value sensor (around e50) effectively detected all experienced malfunctions, in laboratory tests and outdoor deployments, with a few false positives. Future improvements could further lower the cost of our sensor below e10.

show abstract

Section: Discussionmentioning

confidence: 99%

Active Self-Testing Noise Measurement Sensors for Large-Scale Environmental Sensor Networks

Domínguez

Reinoso²,

Touhafi

et al. 2013

Sensors

View full text Add to dashboard Cite

show abstract

“…Therefore, to calibrate Bayesian Networks requires a larger volume of data and/or a higher sampling rate (Bettencourt et al, 2007). Coastal monitoring sensors with fixed positions may satisfy that requirement, allowing for a skillful QC classification (Smith et al, 2012;Rahman et al, 2013Rahman et al, , 2014, so that future improvements for similar scenarios shall come from tuning the Bayesian Network rather than employing a completely different technique. That is not necessarily true for other types of sensors with different sampling strategies.…”

Section: Introductionmentioning

confidence: 99%

A Framework to Quality Control Oceanographic Data

Castelão¹

2020

JOSS

View full text Add to dashboard Cite

Sampling errors are inevitable when measuring the ocean; thus, to achieve a trustable set of observations requires a quality control (QC) procedure capable to detect spurious data. While manual QC by human experts minimizes errors, it is inefficient to handle large datasets and vulnerable to inconsistencies between different experts. Although automatic QC circumvents those issues, the traditional methods results in high rates of false positives. Here, I propose a novel approach to automatically QC oceanographic data based on the anomaly detection technique. Multiple tests are combined into a single, multidimensional criterion that learns the behavior of the good measurements, and identifies bad samples as outliers. When applied to 13 years of hydrographic profiles, the anomaly detection resulted in the best classification performance, reducing the error by at least 50%. An open source Python package, CoTeDe, was developed to provide state of the art tools to quality control oceanographic data.

show abstract

“…However, unlike the previous quality assessment approaches that adopt a single classifier [7][8], we propose an automated algorithm that uses multiple classifiers [12]- [13] to generate quality assessments. Given a set of sensor samples that are each labeled with a quality flag (e.g.…”

Section: Introductionmentioning

confidence: 99%

“…Finally there have been a number of approaches that attempt to learn parameters for data quality classification using labeled examples of the data. Bayesian networks [7] and neural networks [8] are the algorithms that have been used to train the quality assessment system.…”

Section: Introductionmentioning

confidence: 99%

Multiple classifier system for automated quality assessment of marine sensor data

Rahman

Smith

Timms

2013

2013 IEEE Eighth International Conference on Intelligent Sensors, Sensor Networks and Information Processing

View full text Add to dashboard Cite

Numerous sources of uncertainty are associated with the data acquisition process in marine sensor networks. It is thus required to assure that the data quality of sensors is fit for the intended purpose. We propose a supervised learning framework to infer the quality of sensor observations online. A problem with using supervised classification in quality assessment is that sensor observations from the class of uncertain data will be far outweighed by class instances of good data quality. This leads to an imbalanced data set, which can potentially reduce the classification accuracy of uncertain data. A multiple classifier (or ensemble classifier) system is proposed to deal with this problem. Training sets are randomly under-sampled to develop training subsets with balanced class membership. The process is repeated to produce multiple balanced training subsets. Individual classifiers are then trained upon each of these balanced data sets. The quality classifications from the individual classifiers are then combined using majority voting. We evaluated the ensemble classifier system using conductivity and temperature sensors from the Tasmanian Marine Analysis Network (TasMAN). Experiments demonstrate that the ensemble classifier balances the classification accuracy of the majority and minority classes, achieving a higher overall classification accuracy than its constituent classifiers.

show abstract

A Bayesian Framework for the Automated Online Assessment of Sensor Data Quality

Cited by 23 publications

References 34 publications

Active Self-Testing Noise Measurement Sensors for Large-Scale Environmental Sensor Networks

Active Self-Testing Noise Measurement Sensors for Large-Scale Environmental Sensor Networks

A Framework to Quality Control Oceanographic Data

Multiple classifier system for automated quality assessment of marine sensor data

Contact Info

Product

Resources

About