Model and Data Uncertainty for Satellite Time Series Forecasting with Deep Recurrent Models

A Survey of Uncertainty in Deep Neural Networks

“…In this way, once the neural network is trained with dropout layers, the implementation efforts can be kept minimum and the practitioners do not need expert knowledge to reason about uncertainty -certain criteria that the authors are attributing to its success [20]. The practical values of this method has been demonstrated also in several works [152], [10], [21] and resulted in different extensions (evaluating the usage of different dropout masks for example for convolutional layers [153] or by changing the representations of the predictive uncertainty into model and data uncertainties [60]). Approaches that build upon the similar idea but randomly drop incoming activations of a node, instead of dropping an activation for all following nodes, were also proposed within the literature [37] and called drop connect.…”

“…This is primarily because the baseline models are mostly developed using standard data sets such as Cifar10/100, ImageNet, or well known regression data sets that are specific to a particular use case and are therefore not readily applicable to complex real-world environments, as for example low resolutional satellite data or other data sources affected by noise. Although many researchers from other fields apply uncertainty quantification in their field [21], [10], [8], a broad and structured evaluation of existing methods based on different real world applications is not available yet. Works like [56] already built first steps towards a real life evaluation.…”

“…To overcome these limitations, it is essential to provide uncertainty estimates, such that uncertain predictions can be ignored or passed to human experts [20]. Providing uncertainty estimates is not only important for a safe decision-making in high-risks fields, but also crucial in fields where the data sources are highly inhomogeneous and labeled data is rare, such as in remote sensing [21], [22]. Also for fields where uncertainties form a crucial part of the learning techniques, such as for active learning [23], [24], [25], [26] or reinforcement learning [20], [27], [28], [29], uncertainty estimates are highly important.…”

Uncertainty-Guided Representation Learning in Local Climate Zone Classification

“…Furthermore, models based on Bayesian reasoning have been established and UQ based on this framework has been implemented recently with the aid of dropout networks [10]. The method is widely used, although the application to remote sensing data is yet limited [11].…”

Going Beyond One-Hot Encoding in Classification: Can Human Uncertainty Improve Model Performance?

“…Methodological directions to estimate uncertainty quantities include the aggregation of multiple neural networks (Lakshminarayanan et al, 2017), deterministic networks with distributional assumptions placed on the label space (Malinin & Gales, 2018;Sensoy et al, 2018), sophisticated use of dropout networks (Gal & Ghahramani, 2016), or the Bayesian neural networks (Blundell et al, 2015;Maddox et al, 2019). Domain-specific applications in the remote sensing area are still rare (Gawlikowski et al, 2022;Russwurm et al, 2020).…”