Insight. Applied.
People find it difficult to intuitively gauge the level of correlation between variables.
Guess the Correlation is an 80s-style video game that lets you flex you estimation muscles.
Just be aware that it doesn’t seem to present negative correlations, so you’ll have to intuit those elsewhere.
April is Mathematics & Statistics Awareness Month.
Let’s celebrate it by making sure we embrace statistics in our data science projects. It’s not just about Python, folks!
Bill Gates retweeted a World Health Organization infographic showing that mosquitos kill vastly more people than sharks every year—on the order of 100,000 times more.
In his tweet Bill captioned the infographic with, "Why I would rather encounter a shark in the wild rather than a mosquito." Presumably he’s referring to man-eating sharks.
This was an informal comment designed to highlight the misery caused by malaria—a cause that is at the centre of Bill’s philanthropy. Clearly it wasn’t supposed to be a serious risk assessment.
But it illustrates how confusing conditional probabilities are, and how easy it is to make invalid statistic inferences.
The data in the infographic refer to the probability that, given you are dead, you were killed by a shark or a mosquito. Chances are that it was a mosquito—not a shark. That seems intuitive.
Technically, we can denote this as
$P(shark|death) << P(mosquito|death)$
I’m not convinced by Bill’s implication that it’s better to encounter a shark than a mosquito. I grew up after "Jaws" was released. Intuitively, surely sharks are much more dangerous, right?
The risk posed by meeting either of these creatures is the probability of being killed given you met them. If we encountered man-eating sharks as often as we encounter mosquitos we’d be getting munched on constantly.
Sharks are definitely scarier. We can represent this more formally as
$P(death|shark) >> P(death|mosquito)$
It’s important that we distinguish between $P(mosquito|death)$ and $P(death|mosquito)$ when drawing inferences.
Fortunately man-eating sharks live in the ocean and I don’t. Given that, I’m willing to sign up for more killer sharks and less mosquitos.
Statistical confidence intervals are almost always misinterpreted. Consider the following statement.
"The prevalence of the disease P has a 95% confidence interval of 1% <= P <= 5%."
This is commonly taken to imply that there’s a 95% chance that the true prevalence is between 1% and 5%.
This isn’t the case.
Confidence intervals represent uncertainty about the interval, rather than the parameter of interest.
The correct interpretation of the confidence interval defined above is that if we collect many samples from the population and calculate confidence intervals from them, 95% of those confidence intervals will contain the true value of P.
In Bayesian statistics we generally calculate credible intervals which are compatible with the intuitive interpretation.
A statistical note published in the BMJ (formally the British Medical Journal) reminds us that
…trials that do not show a significant difference between the treatments being compared are often called "negative." This term wrongly implies that the study has shown that there is no difference, whereas usually all that has been shown is an absence of evidence of a difference. These are quite different statements.