Probabilistic Random Forest: A Machine Learning Algorithm for Noisy Data Sets

Abstract

Machine learning (ML) algorithms become increasingly important in the analysis of astronomical data. However, since most ML algorithms are not designed to take data uncertainties into account, ML based studies are mostly restricted to data with high signal-to-noise ratio. Astronomical datasets of such high-quality are uncommon. In this work we modify the long-established Random Forest (RF) algorithm to take into account uncertainties in the measurements (i.e., features) as well as in the assigned classes (i.e., labels). To do so, the Probabilistic Random Forest (PRF) algorithm treats the features and labels as probability distribution functions, rather than deterministic quantities. We perform a variety of experiments where we inject different types of noise to a dataset, and compare the accuracy of the PRF to that of RF. The PRF outperforms RF in all cases, with a moderate increase in running time. We find an improvement in classification accuracy of up to 10% in the case of noisy features, and up to 30% in the case of noisy labels. The PRF accuracy decreased by less then 5% for a dataset with as many as 45% misclassified objects, compared to a clean dataset. Apart from improving the prediction accuracy in noisy datasets, the PRF naturally copes with missing values in the data, and outperforms RF when applied to a dataset with different noise characteristics in the training and test sets, suggesting that it can be used for Transfer Learning.

Classifying exoplanet candidates with convolutional neural networks: application to the Next Generation Transit Survey

“…Interestingly however, we find that our networks are robust to contaminated labels; only minor degradation in overall performance is experienced up to a contamination fraction of 0.48, after which performance decreases rapidly. This result is consistent with those from other studies (Rolnick et al 2017;Li et al 2019;Reis et al 2019) and suggests that label contamination in real data is of little consequence to overall performance.…”

“…The loss in accuracy up to 45% contamination was ∼4%. Reis et al (2019) perform the same experiment for probabilistic random forests and found a loss of less than 5% when more than 45% of their dataset had incorrect labels, in-line with the performance drop we find. Evidently label contamination does hinder performance, but the network is robust to small contamination fractions.…”

Photometric redshifts for X-ray-selected active galactic nuclei in the eROSITA era

“…Currently, in our experiment, about 35% are lacking a photo-z for this reason. However, the use of fluxes instead of magnitudes and adding the photometric errors to the list of parameters (using e.g., Reis et al 2019) should solve the issue.…”

“…The photometric catalogues used in Stripe 82X are relatively shallow, implying that the fainter sources in general have a large photometric error. Unlike SED-fitting, until recently ML techniques could not handle the errors associated to the measurements and the same weight was incorrectly assumed for each photometric value (but see Reis et al 2019, for a counter example application). We tried to assess how this is impacting on the result, by reducing BE ST magcolopt to subsamples of decreasing photometric errors in all the bands.…”

Blind chemical tagging with DBSCAN: prospects for spectroscopic surveys

“…To obtain better recovery fractions using spectra directly, better methods to compute the distance between spectra will be necessary (e.g. Reis et al 2019).…”