Robust Learning Through Cross-Task Consistency

Abstract

Visual perception entails solving a wide set of tasks, e.g., object detection, depth estimation, etc. The predictions made for multiple tasks from the same image are not independent, and therefore, are expected to be 'consistent'. We propose a broadly applicable and fully computational method for augmenting learning with Cross-Task Consistency. 1 The proposed formulation is based on inference-path invariance over a graph of arbitrary tasks. We observe that learning with cross-task consistency leads to more accurate predictions and better generalization to out-of-distribution inputs. This framework also leads to an informative unsupervised quantity, called Consistency Energy, based on measuring the intrinsic consistency of the system. Consistency Energy correlates well with the supervised error (r=0.67), thus it can be employed as an unsupervised confidence metric as well as for detection of out-of-distribution inputs (ROC-AUC=0.95). The evaluations are performed on multiple datasets, including Taskonomy, Replica, CocoDoom, and ApolloScape, and they benchmark cross-task consistency versus various baselines including conventional multi-task learning, cycle consistency, and analytical consistency.

Robustness via Cross-Domain Ensembles

“…Epistemic uncertainty accounts for uncertainty in the model parameters, while aleatoric uncertainty stems from the noise inherent in the data. There are many proposed methods to estimate the former, such as using dropout [12,46], stochastic variational inference methods [5,14,37,36,50,42], ensembling [32], and consistency energy [57] where a single uncalibrated uncertainty estimate is extracted from consistency of different paths. Most of the existing methods in this area solely estimate uncertainty without using it towards improving the predictions.…”

“…Enforcing consistency constraints in the context of cross-task predictions involves ensuring that the output predictions remain the same regardless of the intermediate domain [57,34,60,53]. Particularly in contrast to [57] which uses (non-probabilistic) training-time consistency constraints to improve a network's prediction and does not have any consolidation mechanism, our goal is to robustify the final prediction by merging the output of multiple prediction paths at the test time. Our formulation and the training-time consistency constraints are complimentary.…”

“…where N is the number of samples, and z n is the label for the nth sample. This results in an 1 -norm loss on the errors as opposed to an 2 -norm loss with a Gaussian distribution, which has been shown to improve prediction quality [27,57]. Finally, the sigma is given by √ 2 exp(ŝ n ) and it captures the uncertainty in predictions.…”

“…We also augment the network training with "cross-task consistency constraints" (X-TC) [57] for generally better results, but this is not a fundamental requirement (ablation results provided in Sec. 4.1.3).…”