Predicting organic acid concentration from UV/vis spectrometry measurements – a comparison of machine learning techniques

Abstract

The concentration of organic acids in anaerobic digesters is one of the most critical parameters for monitoring and advanced control of anaerobic digestion processes. Thus, a reliable online-measurement system is absolutely necessary. A novel approach to obtaining these measurements indirectly and online using UV/vis spectroscopic probes, in conjunction with powerful pattern recognition methods, is presented in this paper. An UV/vis spectroscopic probe from S::CAN is used in combination with a custom-built dilution system to monitor the absorption of fully fermented sludge at a spectrum from 200 to 750 nm. Advanced pattern recognition methods are then used to map the non-linear relationship between measured absorption spectra to laboratory measurements of organic acid concentrations. Linear discriminant analysis, generalized discriminant analysis (GerDA), support vector machines (SVM), relevance vector machines, random forest and neural networks are investigated for this purpose and their performance compared. To validate the approach, online measurements have been taken at a full-scale 1.3-MW industrial biogas plant. Results show that whereas some of the methods considered do not yield satisfactory results, accurate prediction of organic acid concentration ranges can be obtained with both GerDA and SVM-based classifiers, with classification rates in excess of 87% achieved on test data.

Tuned data mining

“…Thus the benchmark is also indirectly cost-sensitive although the gain matrix is the unit matrix in this case. The research question here is whether classification methods based on TDM can achieve a similar or even better performance than the GerDA results reported in [28]. GerDA, as described in [25,29], learns unsupervisedly interesting feature combinations with an approach based on Boltzmann machines (it has to be noted that in [28] the classifier superimposed on the GerDA features was optimized for the overall misclassification rate instead of MCA).…”

“…Acid concentrations in the fluid of a plant are to be classified in this benchmark, based solely on spectroscopy data [28]. In the appAcid task there are five defined classes, each denoting a certain range of acid concentration.…”

“…The TDM approach presented in this study, which uses generic feature-processing and -selection methods, performs competitive to GerDA [25,28]. GerDA implements a very sophisticated feature generation approach.…”

“…Each cell contains mean ± standard deviation from 5 repeated runs with different random seeds. The "Rank DMC" column a/b is the rank a of TST-result within the real DMC result [28] 87.2 off the extra feature generation (monomials) gives a 3.2% decrease in accuracy, turning off PCA gives a larger decrease (6.5%), but the largest decrease in accuracy (7.3%) occurs if we turn off the feature selection (FS), see line 6 in Tab. 7.…”

Efficient multi-criteria optimization on noisy machine learning problems

“…AppAcid is a dataset for classifying UV/vis spectrography measurements of a biogas plant [60]. It contains C = 5 classes and the class patterns are unevenly distributed.…”