"Statistical Results Associated With Increased Risk Of Exaggerating Risk"

From William Briggs, Statistician to the Stars:

Our title, which is indistinguishable from a flood of others¹, might read, “Reading Articles About The Misuse Of Statistics Increases Risk Of Apoplexy.”

Yes, for every article you read like this one, your risk of becoming apoplectic over the improper use of statistics increases 2.0 times.

What does that “2.0-fold increase in risk” mean? Not just for this finding, but for any which reports results in the form of “increased risk” of suffering from a malady after being exposed to some “risk factor.” In this study, “exposure” is reading this blog, which is the risk factor, and “non-exposure” is not reading.

Suppose (somehow) you knew the probability of developing the malady given you were not exposed to the “risk factor.” Call it prob_{not exposed}. You also have to know (somehow) the probability of developing the malady given you were exposed; called prob_exposed. Relative risk is

     RR = prob_exposed / prob_{not exposed}.
You could also calculated the odds ratio. First know that odds are a one-to-one function of probability, viz:
     Odds = prob / (1 – prob).
The odds ratio is like the risk ratio, but the ratio of the odds, not probabilities:
     OR = odds_exposed / odds_{not exposed}.

Now suppose that prob_{not exposed} = 0.000001, which is a one in a million chance of developing the malady given you were not exposed. If you then hear that being exposed “increases the risk by 2.0 fold”, then this means the risk ratio must be 2.0. Back solving gives the probability of developing the malady after exposure as 0.000002. (Similar calculations can be done for odds ratios.)

In this case, exposure drove your risk from one in a million to just 2 in a million. We can already see that presenting results in raw probability will not be as pulse pounding as speaking in terms of risk or odds. Information is also lost in giving the risk ratio: the customer has no idea what the risk is in the control group. So one fix would be to give emphasis to the actual probabilities of suffering, and not just the risk ratio.

But even if that is done, something would still be wrong. Can you spot what?...MORE