Decision Mechanics

Insight. Applied.

What the hell is “data science” anyway?!

By

Too much time has been squandered on trying to define data science, AI, machine learning, data analysts, data thingymajig, etc.

Who cares? I’m happy to return to “statistics” and “programming”.

We should follow the lead of United States Supreme Court Justice Potter Stewart.

I shall not today attempt further to define the kinds of material I understand to be embraced within that shorthand description [“hard-core pornography”], and perhaps I could never succeed in intelligibly doing so. But I know it when I see it…

Confidence intervals

By

Statistical confidence intervals are almost always misinterpreted. Consider the following statement.

"The prevalence of the disease P has a 95% confidence interval of 1% <= P <= 5%."

This is commonly taken to imply that there’s a 95% chance that the true prevalence is between 1% and 5%.

This isn’t the case.

Confidence intervals represent uncertainty about the interval, rather than the parameter of interest.

The correct interpretation of the confidence interval defined above is that if we collect many samples from the population and calculate confidence intervals from them, 95% of those confidence intervals will contain the true value of P.

In Bayesian statistics we generally calculate credible intervals which are compatible with the intuitive interpretation.

Pandas in 8 pages

By

Pandas is a Python package for working with tabular data—as in a spreadsheet or database table. It provides similar functionality to R’s data frames.

As pandas is rich in features it can be difficult to remember all its operations and syntax, so Enthought have produced a visual guide to the package in 8 handy pages.

Is it easier to learn R or Python?

By

Andy Kirk at Visualising Data ran a Twitter poll about the relative accessibility of R and Python to non-developers.

59% said that R was more accessible.

Obviously, the poll is far from scientific, but the comments he received reflect my own experiences of teaching both languages—such as the significance of the RStudio IDE and the tidyverse packages in getting people off the ground.