How Accurate Are Expert Estimations of Correlation?

Abstract

In many practical situations, it is important to know the correlation between different quantities -finding correlations helps to gain insights into various relationships and phenomena, and helps to inform analysts. Often, there is not enough empirical data to experimentally determine all possible correlations. In such cases, a natural idea is to supplement this situation with expert estimates. Expert estimates are rather crude. So, to decide whether to act based on these estimates, it is desirable to know how accurate are expert estimates. In this paper, we propose several techniques for gauging this accuracy.