Calibration of Machine Reading Systems at Scale

Abstract

In typical machine learning systems, an estimate of the probability of the prediction is used to assess the system's confidence in the prediction. This confidence measure is usually uncalibrated; i.e. the system's confidence in the prediction does not match the true probability of the predicted output. In this paper, we present an investigation into calibrating open setting machine reading systems such as opendomain question answering and claim verification systems. We show that calibrating such complex systems which contain a discrete retrieval and deep reading components is challenging and current calibration techniques fail to scale to these settings. We propose simple extensions to existing calibration approaches that allow us to adapt these callibrators to these settings.Our experimental results reveal that the joint callibration of the retriever and the reader outperforms the reader calibrator by a significant margin. We also show that the callibrator can be useful for selective prediction, e.g., when question answering systems are posed with unanswerable or out-of-the-training distribution questions.

A Diachronic Perspective on User Trust in AI under Uncertainty

“…A common method AI systems use to convey their uncertainty to the user is by its confidence (Benz and Rodriguez, 2023;Liu et al, 2023). For the system's confidence to reflect the probability of the system being correct, the confidence needs to be calibrated, which is a long-standing task (Guo et al, 2017;Dhuliawala et al, 2022). This can be any metric, such as quality estimation (Specia et al, 2010;Zouhar et al, 2021) that makes it easier for the user to decide on the AI system's correctness.…”