From the Federal Reserve Bank of New York's Liberty Street Economics blog:
Uncertainty is of considerable interest for understanding the behavior
of individuals as well as the movements in key macroeconomic and
financial variables. Despite its importance, direct measures of
uncertainty aren’t widely available. Because of this data limitation, a
common practice is to use survey-based measures of forecast
dispersion—reflecting disagreement among respondents—to proxy for
uncertainty. Is this a reliable practice? Here, we review the
distinction between disagreement and uncertainty as concepts, and show
that this conceptual distinction carries over to their empirical
counterparts, suggesting that disagreement is not generally a good proxy
for uncertainty.
The terms “disagreement” and “uncertainty” refer to very different
concepts. Disagreement refers to a collection of forecasts or point
predictions and the nature of their clustering around each other—the
more disperse the forecasts, the greater the extent of disagreement
among the survey respondents. On the other hand, uncertainty refers to
the distribution of the probabilities that a respondent attaches to the
different possible outcomes of the forecasted variable—the more
confidence held by a respondent, the tighter this distribution is and
the lower the respondent’s uncertainty.
Although disagreement and uncertainty are different concepts, some
commentators have drawn a connection between the two. That is, episodes
characterized by high (low) disagreement are viewed as indicative of
high (low) uncertainty shared by the respondents. This assumption
provides the basis to use disagreement as a proxy for uncertainty when
measures of the latter magnitude aren’t available.
Is the assumed positive association between disagreement and uncertainty
plausible? Yes. Is it necessarily true? No. To understand why either
case is possible, we can look at the following figure previously
discussed in this paper by Victor Zarnowitz and Louis Lambros:
The illustration on the left shows the forecasts and associated
probability distributions of two hypothetical survey
respondents—respondent A and respondent B. The close proximity of the
forecasts (ŷA and ŷB) indicates
low disagreement, while the tight distributions around each forecast
indicate low uncertainty. However, the illustration on the right depicts
another possible situation. Here the forecasts are unchanged, so
disagreement remains low. But the probability distributions around each
forecast are now much wider, indicating high uncertainty. The figure is
important for two reasons. First, it bears directly on the question of
the reliability of disagreement as a proxy for uncertainty. While the
conditions depicted in the left illustration might justify this
practice, the conditions depicted in the right illustration would not.
Second, it shows that the dispersion of forecasts by itself is not
necessarily informative about the level of uncertainty across
respondents. Because most surveys only report the respondents’
forecasts, observing the degree to which the forecasts cluster together
can’t tell you whether the illustration on the left or on the right in
the figure is the more relevant situation....MORE
